-- Logs begin at Fri 2022-06-24 09:30:49 UTC, end at Fri 2022-06-24 10:38:10 UTC. -- Jun 24 10:01:54.351775 dv-cinder-847132-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 10:01:54.360353 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 10:01:54 2022] *** Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: nodename: dv-cinder-847132-2-iscsi Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: machine: x86_64 Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: clock source: unix Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: pcre jit disabled Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: detected number of CPU cores: 4 Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: current working directory: / Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: detected binary path: /usr/bin/uwsgi-core Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: your processes number limit is 47680 Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: your memory page size is 4096 bytes Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: detected max file descriptor number: 2048 Jun 24 10:01:54.362358 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: lock engine: pthread robust mutexes Jun 24 10:01:54.363226 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: thunder lock: enabled Jun 24 10:01:54.363226 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 10:01:54.365630 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 10:01:54.407804 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: Python main interpreter initialized at 0x55b4c56ed9c0 Jun 24 10:01:54.407804 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: python threads support enabled Jun 24 10:01:54.407804 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: your server socket listen backlog is limited to 100 connections Jun 24 10:01:54.407804 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: your mercy for graceful operations on workers is 90 seconds Jun 24 10:01:54.408223 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 10:01:54.408296 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: *** Operational MODE: preforking *** Jun 24 10:01:54.408569 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: *** uWSGI is running in multiple interpreter mode *** Jun 24 10:01:54.408569 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: spawned uWSGI master process (pid: 80283) Jun 24 10:01:54.408799 dv-cinder-847132-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 10:01:54.410180 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: spawned uWSGI worker 1 (pid: 80295, cores: 1) Jun 24 10:01:54.410180 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: spawned uWSGI worker 2 (pid: 80296, cores: 1) Jun 24 10:01:54.410180 dv-cinder-847132-2-iscsi devstack@keystone.service[80283]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 10:01:59.883906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:01:59.883906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: from cryptography.utils import int_from_bytes Jun 24 10:01:59.884744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:01:59.884744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: from cryptography.utils import int_from_bytes Jun 24 10:01:59.886176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:01:59.886176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: from cryptography.utils import int_from_bytes Jun 24 10:01:59.886668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 10:01:59.886668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: from cryptography.utils import int_from_bytes Jun 24 10:01:59.933295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 10:01:59.933509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 10:01:59.933671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] command line args: [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 10:01:59.933932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 10:01:59.934054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 10:01:59.934295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ================================================================================ {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 10:01:59.934383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 10:01:59.934446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] admin_token = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.934664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] command line args: [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 10:01:59.934821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] config_dir = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.935168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.935317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 10:01:59.935407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] config_source = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.935487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ================================================================================ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 10:01:59.935548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] debug = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.935710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.935963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] admin_token = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.935963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_dir = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_source = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] debug = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.936946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_config_append = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_dir = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_file = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.937966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_config_append = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_dir = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.938956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.939889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.940987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] publish_errors = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.941905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] publish_errors = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.942952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] standard_threads = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] standard_threads = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] use_journal = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.943943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] use_json = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] use_stderr = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] use_syslog = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_journal = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_json = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.944773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_stderr = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_syslog = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.944977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 10:01:59.945212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.945936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.946910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.947850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.948949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.949744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.950959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.951930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.952847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.953948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.954949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.955852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.956992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.957963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.958944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.959937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.960863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.961956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.962914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.963923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.964848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.965975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966443 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.966930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.967918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.968998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.external = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.969949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.password = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] auth.token = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.external = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.970981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.password = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.token = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.971928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.972863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973557 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.973944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.974993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975402 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.975969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.976961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.977900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.978877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.979028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.979107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.979277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.979357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.979876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.979963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.980894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.981086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.981169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.981339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.981591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.981824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.981915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.982888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.983861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.984944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.985979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.986938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.987991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.988925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.989925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.990888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.991962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.992986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.993907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.994872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.995935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.996813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.997877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.998987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:01:59.999934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000718 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.000955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.001902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] role.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.002865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] role.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.003949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.004971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.005918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.006992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.007970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.008929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.009904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.010900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.011855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.012001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.012081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.012275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.012358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.012547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.012784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.013971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.014921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.connection = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.015998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.016920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.017860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.018997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.019938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.020016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.020110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80295) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 10:02:00.020190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.020401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.020633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.020873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.021104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.021348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.021580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.021821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.022047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.022283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.022511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 10:02:00.022678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 10:02:00.051327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.051538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.051766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.053278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.053509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.053731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.060587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.063004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.088688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.088897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.089390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.089633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.237220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.239936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.285672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.285859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.287093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.287306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.349764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.349967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.350918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.351079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.351293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.351594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.376309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.376500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.376745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.376872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.376946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.377966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.378152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.388767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.389781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.401600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.401758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.401854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.401921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.401998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.402995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80295) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 10:02:00.420358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.420571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:02:00.420697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.420859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:02:00.420967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.421049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:02:00.421200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.421292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.421399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 10:02:00.421569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.421710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.421794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.421898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.422034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.422169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.422413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.432958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.433990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.442241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.442511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.445569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.445842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.445969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.446974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.447998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.448181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.448308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.448440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.475639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.477250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.477535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.479013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.481049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.481832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.483719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.484120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.484398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.487169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.487622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.487808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.491843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.492129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.495146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.495398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.496678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.496964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.497244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.497521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.497793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.498062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.498327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.498594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.500170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.500455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.500703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.500999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.501265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.501533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.501793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.502039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.517532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.517810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.522452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.524833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.525103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.526124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.526514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.530270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.531623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.533898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.534294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.535271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.538977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.539262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.539405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.539522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.539826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.540102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.540370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.542955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.547181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.547472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.547752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.548031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.548317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.548588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.560108 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.560392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.560656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.560931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.568258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.568539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.568802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.568928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.569044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.570591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.570897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.574077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.574361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.574638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.575004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.575358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.577240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.578971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.579252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.582422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.582697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.582996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.583277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.583564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.588666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.588920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.589226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.589504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.591249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.591539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.591828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.597340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.597618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.597904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.598176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.599884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.600174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.600459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.610204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.613561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.616903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.618643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.619270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.620307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.620587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.620856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.623094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.626414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.628152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.628279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.629778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.629927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.630002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.630297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.633123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.633408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.633797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.634074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.637743 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.639496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.642739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.643030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.643409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.643539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.643665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.643758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.643964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.651950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80295) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.652229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.652509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.652777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.653080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.653200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.653273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.653431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.653719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.653845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.654001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.654266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.654535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.654828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.655094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.655357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.655623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80295) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.661746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 10:02:00.662022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.662290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.662553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.662838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.663220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.663497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.663756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.664022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.664290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.664554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.664818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.665090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.665343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 10:02:00.693809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b4c56ed9c0 pid: 80295 (default app) Jun 24 10:02:00.697421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68699fc-4e21-4989-8290-edb9298f3897 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:00.697717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68699fc-4e21-4989-8290-edb9298f3897 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:00.698004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68699fc-4e21-4989-8290-edb9298f3897 None None] PATH_INFO: `/v3` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:00.702308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Fri Jun 24 10:02:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:00.704431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b4c56ed9c0 pid: 80296 (default app) Jun 24 10:02:08.521311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:08.522289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:08.522289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:08.524335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 2/2] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:08.532580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:08.533033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:08.533123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:08.601438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: INFO dbcounter [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] Registered counter for database keystone Jun 24 10:02:08.625627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG oslo_db.sqlalchemy.engines [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80296) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 10:02:08.636861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writer thread running {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 10:02:08.824385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG passlib.handlers.bcrypt [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80296) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 10:02:08.824701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG passlib.handlers.bcrypt [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80296) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 10:02:08.881209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:08.882854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:09.021356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1/3] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 10:02:09.165349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:09.238464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: INFO dbcounter [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] Registered counter for database keystone Jun 24 10:02:09.260959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG oslo_db.sqlalchemy.engines [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80295) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 10:02:09.272811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writer thread running {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 10:02:09.395429 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcd742-29df-4e70-869c-86457476f9f8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:09.401724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:09.402685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:09.403215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:09.403501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:09.412946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:09.464719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 10:02:09.465011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753ec8c-35bc-4dfe-a425-0d7fffe03fed admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:09.476159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 3/4] 10.222.0.64 () {58 vars in 1324 bytes} [Fri Jun 24 10:02:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:02:09.483853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:09.508830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd2b33-0f3e-4414-a2a9-49628ce13843 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:09.517522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:09.519280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:09.519655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:09.519941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] PATH_INFO: `/v3/projects/98caa12ca1b14e52b73e3cce22837778` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:09.526596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] RBAC: Authorizing `identity:get_project(project_id=98caa12ca1b14e52b73e3cce22837778)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:09.584963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING keystone.common.rbac_enforcer.enforcer [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 10:02:09.585578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da376333-4c57-4b19-af0f-efb926b6bddf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:09.590134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 2/5] 10.222.0.64 () {56 vars in 1207 bytes} [Fri Jun 24 10:02:09 2022] GET /identity/v3/projects/98caa12ca1b14e52b73e3cce22837778 => generated 344 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:11.652158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ba91-1d47-406f-a7d9-ce93812fc74a None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:11.652759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ba91-1d47-406f-a7d9-ce93812fc74a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:11.652759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ba91-1d47-406f-a7d9-ce93812fc74a None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:11.658406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 4/6] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:11.665408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:11.665897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:11.665984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:11.688250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:11.689798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:11.699734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 3/7] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 10:02:11.723388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075ba91-1d47-406f-a7d9-ce93812fc74a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:11.739536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075ba91-1d47-406f-a7d9-ce93812fc74a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:11.744414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfbf63-a4a1-4c89-960c-608197269ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:11.745072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfbf63-a4a1-4c89-960c-608197269ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:11.745349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfbf63-a4a1-4c89-960c-608197269ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:11.745633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfbf63-a4a1-4c89-960c-608197269ea7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:11.754596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfbf63-a4a1-4c89-960c-608197269ea7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:11.756439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfbf63-a4a1-4c89-960c-608197269ea7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:11.766370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 5/8] 10.222.0.64 () {58 vars in 1324 bytes} [Fri Jun 24 10:02:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:02:11.773287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:11.783440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118b5da-0a69-4e9d-b9c5-aeead75d83e1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:11.789842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bbbef-0635-4f80-8e7b-313fca424b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:11.790741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bbbef-0635-4f80-8e7b-313fca424b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:11.791033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bbbef-0635-4f80-8e7b-313fca424b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:11.791345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bbbef-0635-4f80-8e7b-313fca424b00 admin admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:11.792942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bbbef-0635-4f80-8e7b-313fca424b00 admin admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:11.794866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bbbef-0635-4f80-8e7b-313fca424b00 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:11.796872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 4/9] 10.222.0.64 () {56 vars in 1198 bytes} [Fri Jun 24 10:02:11 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:13.910149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceddd63-d9e4-45f9-91d1-a69d1ac35280 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:13.910769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceddd63-d9e4-45f9-91d1-a69d1ac35280 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:13.910769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceddd63-d9e4-45f9-91d1-a69d1ac35280 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:13.912113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 6/10] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:13.917382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:13.917841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:13.917922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:13.935592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:13.936869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:13.943674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee845d5-c962-4cde-b1b1-e61558a2a44e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:13.943962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee845d5-c962-4cde-b1b1-e61558a2a44e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:13.944236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee845d5-c962-4cde-b1b1-e61558a2a44e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:13.945817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 7/11] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:13.951655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14172e-a413-420f-9166-b541fc68f162 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:13.951959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14172e-a413-420f-9166-b541fc68f162 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:13.952259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14172e-a413-420f-9166-b541fc68f162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:13.994239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG passlib.handlers.bcrypt [None req-da14172e-a413-420f-9166-b541fc68f162 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80295) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 10:02:13.994817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG passlib.handlers.bcrypt [None req-da14172e-a413-420f-9166-b541fc68f162 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80295) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 10:02:14.023434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 5/12] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:14.039776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-da14172e-a413-420f-9166-b541fc68f162 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:14.041410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-da14172e-a413-420f-9166-b541fc68f162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:14.047993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.074341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.147255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 8/13] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:14.153954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbb580-963d-4c26-95ce-58a6bd0a9182 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.169861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14172e-a413-420f-9166-b541fc68f162 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.188626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.189437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.189725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.190000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.202616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:14.213997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:14.214946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:14.215760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 6/14] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:14.221332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.274416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14172e-a413-420f-9166-b541fc68f162 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.276995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.309007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.316141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.317177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.317469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.317751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.328897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a981193-46d8-491c-86ef-83846468ebdb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.365639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.366357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.366647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.366951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.367791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:14.368059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.369605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.383994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 7/15] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:14.390739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.391878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.392648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.499265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40138212-6d36-42e7-978e-3f87987a57f6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.501061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 9/16] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:02:14.507168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.510131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.542591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.543412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.543694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.543958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.545253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.546800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.548672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 8/17] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:14.553960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.570620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.616219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec86140-4a66-443c-9b7d-62fae2419302 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.651049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.651856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.652163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.652438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.658348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec7df0-befd-4d67-b4f7-6c25da6c5267 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.673093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.674592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.692911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.693723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.694012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.694292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:02:14.695358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:14.696277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:14.697122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:02:14.697122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:02:14.697122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:02:14.697122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:14.697343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.697413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 10/18] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:14.697476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.698673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 9/19] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:14.702735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.706801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:14.765520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.795106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:14.815807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e502918-872a-4e0c-add0-1b86ccc1a89b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.816247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b40e8-5029-425a-af7f-67a36de93cbe None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:14.851835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.852387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:14.852499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.852773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.853045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.853134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:14.853346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:14.853630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:14.853903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.domain.id=default, user.id=68ba5f09c78c41f18a7e1c6789016a58) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:14.854216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.854373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:14.855080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:14.856113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfecf60-4088-4cff-88e9-fa96ec0b766f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.856473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2a6a-fd04-43b0-9d42-80c7a461c6c8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:14.865977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 10/20] 10.222.0.64 () {56 vars in 1322 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/role_assignments?user.id=68ba5f09c78c41f18a7e1c6789016a58&scope.domain.id=default&role.id=b10ab4271e1845e4aefab70d82a12def => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:14.876473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 11/21] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:17.082365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63703659-442f-404a-ab6e-b2f5d1900880 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.082896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63703659-442f-404a-ab6e-b2f5d1900880 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.082896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63703659-442f-404a-ab6e-b2f5d1900880 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.084443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 11/22] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:17.090105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.090941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.091369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.120409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:17.121873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:17.213142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 12/23] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:17.255874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:17.256179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e805921-0f1e-4391-856c-05aad8f8c385 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.256429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e805921-0f1e-4391-856c-05aad8f8c385 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.256714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e805921-0f1e-4391-856c-05aad8f8c385 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.258273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 12/24] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:17.264286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.264574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.264861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.295777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:17.297131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:17.333017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:17.406501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f2a0-63d1-4c88-bdd9-63acafce8069 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:17.444127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 13/25] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 10:02:17.467868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:17.468899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.469189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.469468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.484593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:17.486003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:17.503035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:17.503952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:17.504691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 13/26] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:02:17.512543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:17.630354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:17.666586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5df57a-17fa-4b08-a060-4c712bfadda6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:17.693569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e7fa0-7984-47ee-ba48-73af9f065dba None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:17.709744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:17.751569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3532a-b78d-41b2-a3e3-f44d19d97346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:17.752289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3532a-b78d-41b2-a3e3-f44d19d97346 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.752573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3532a-b78d-41b2-a3e3-f44d19d97346 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.752852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3532a-b78d-41b2-a3e3-f44d19d97346 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.753647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3532a-b78d-41b2-a3e3-f44d19d97346 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:17.755135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3532a-b78d-41b2-a3e3-f44d19d97346 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:17.773135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147697f-6431-446c-b801-302654b98a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:17.774120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147697f-6431-446c-b801-302654b98a0d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.774407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147697f-6431-446c-b801-302654b98a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.774683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147697f-6431-446c-b801-302654b98a0d None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.775593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147697f-6431-446c-b801-302654b98a0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:17.778150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147697f-6431-446c-b801-302654b98a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:17.780481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147697f-6431-446c-b801-302654b98a0d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:17.802141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 14/27] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:17.803397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08838b-3264-4533-b80f-33853cf13239 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.803674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08838b-3264-4533-b80f-33853cf13239 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.803958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08838b-3264-4533-b80f-33853cf13239 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.805241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 15/28] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:17.809894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08838b-3264-4533-b80f-33853cf13239 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:17.900607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 14/29] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Jun 24 10:02:17 2022] POST /identity/v3/roles => generated 219 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:17.902338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21067d13-fc51-4559-8089-0e20f46eb5b7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:17.902636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21067d13-fc51-4559-8089-0e20f46eb5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:17.902951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21067d13-fc51-4559-8089-0e20f46eb5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:17.933412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-21067d13-fc51-4559-8089-0e20f46eb5b7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:17.935854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-21067d13-fc51-4559-8089-0e20f46eb5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:18.015814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 15/30] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:18.017533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff2f2c-f768-49ff-a881-f1bf10dcbce8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.018098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff2f2c-f768-49ff-a881-f1bf10dcbce8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.018569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff2f2c-f768-49ff-a881-f1bf10dcbce8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.020231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 16/31] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:18.025892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.026474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.027079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.042274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08838b-3264-4533-b80f-33853cf13239 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.045261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:18.046843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:18.115711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.117318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.117791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.118256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.120986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.122790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.126375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 16/32] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:17 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:18.127442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:18.138374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:18.140772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 17/33] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:18.143626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:18.188687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:18.275968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544bb3-c079-4283-a59f-3ac746d81964 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.288885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b0f1-66b3-481f-823e-bb8c5d41d47d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.322128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.323380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.323867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.324342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.337879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.339198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.339303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.339802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.340273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.340806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.341990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.344055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.346046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 18/34] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:18.347368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:02:18.361092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:18.362002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:18.363282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:02:18.363282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:02:18.363282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:02:18.363282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:18.363282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 17/35] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:18.363282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:18.419524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:18.463901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38754b-995a-45a7-941e-95c5fd19b4e4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.490598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:18.503522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.504584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.505088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.505565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.506555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.506821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a0399-033d-4b56-bebf-49313640d5e8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.508507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.526344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 19/36] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Jun 24 10:02:18 2022] POST /identity/v3/roles => generated 225 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:18.527645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:18.567686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.568705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.569015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.569276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] PATH_INFO: `/v3/domains/default/users/68ba5f09c78c41f18a7e1c6789016a58/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.574359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=68ba5f09c78c41f18a7e1c6789016a58, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.575428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.653611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf280e4-8e91-40d4-b42b-7c6245f64cd4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.681084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:18.701935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.708475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.709019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.709530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.710596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:18.711126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.712797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e052e-ce52-4e04-bfba-e0d49d429bdf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.735176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 18/37] 10.222.0.64 () {58 vars in 1361 bytes} [Fri Jun 24 10:02:18 2022] PUT /identity/v3/domains/default/users/68ba5f09c78c41f18a7e1c6789016a58/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:18.741221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 20/38] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:18.743647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:18.867745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843119-2bbe-4ebb-9e97-14e5a5be0cc7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.868550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843119-2bbe-4ebb-9e97-14e5a5be0cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.868911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843119-2bbe-4ebb-9e97-14e5a5be0cc7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.870788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 21/39] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:18.877876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.878242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.878594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.880416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71802392-b5c8-4999-970b-ce8c09da40b3 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:18.928845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c8fcc-b8fe-4e71-b571-3120dbcd931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:18.929777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c8fcc-b8fe-4e71-b571-3120dbcd931c None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:18.930184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c8fcc-b8fe-4e71-b571-3120dbcd931c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:18.930526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c8fcc-b8fe-4e71-b571-3120dbcd931c None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:18.931914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c8fcc-b8fe-4e71-b571-3120dbcd931c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:18.933242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c8fcc-b8fe-4e71-b571-3120dbcd931c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:18.970789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:18.981652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 19/40] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:02:18 2022] POST /identity/v3/projects => generated 301 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:02:19.034543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:19.036035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:19.072774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591854-b812-4ee1-ba59-6d127885f00d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:19.073220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591854-b812-4ee1-ba59-6d127885f00d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:19.073580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591854-b812-4ee1-ba59-6d127885f00d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:19.075119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 20/41] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:19.080915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:19.081204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:19.082072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:19.101026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:19.102870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:19.121790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 22/42] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:19.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:19.185215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 21/43] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:19.209767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:19.278108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bc70-5705-402c-8e5b-b02450b90a0f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:19.293329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:19.319489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:19.320325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:19.320700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:19.320937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:19.322258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:19.323868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:19.325723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 23/44] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:19.332436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:19.341417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94bc1c-3471-4787-89d0-1448ab1e953d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:19.374568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:19.386616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e74f-0a50-45b0-ab00-e8ee296650a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:19.387926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e74f-0a50-45b0-ab00-e8ee296650a6 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:19.388353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e74f-0a50-45b0-ab00-e8ee296650a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:19.388989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e74f-0a50-45b0-ab00-e8ee296650a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:19.390562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e74f-0a50-45b0-ab00-e8ee296650a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:19.392215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e74f-0a50-45b0-ab00-e8ee296650a6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:19.408742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 22/45] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Jun 24 10:02:19 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:19.455824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293e5e7-ad5e-4b3a-81d6-ec3dfbb7d1d6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:19.502481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc5006-6083-40ca-8017-4330d5460880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:19.503436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc5006-6083-40ca-8017-4330d5460880 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:19.503941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc5006-6083-40ca-8017-4330d5460880 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:19.504382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc5006-6083-40ca-8017-4330d5460880 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:19.505255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc5006-6083-40ca-8017-4330d5460880 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:19.507065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc5006-6083-40ca-8017-4330d5460880 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:19.539644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 24/46] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:02:19 2022] POST /identity/v3/projects => generated 312 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:20.892687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977add7-54fc-4954-acc1-042685e2a7a0 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:20.893211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977add7-54fc-4954-acc1-042685e2a7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:20.893294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977add7-54fc-4954-acc1-042685e2a7a0 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:20.894727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 23/47] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:20.900209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:20.900599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:20.900782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:20.920857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:20.922081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:20.950647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:20.995024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 25/48] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:21.018451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977add7-54fc-4954-acc1-042685e2a7a0 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:21.127727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977add7-54fc-4954-acc1-042685e2a7a0 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:21.163343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:21.164154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.164443 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.164729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.176067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:21.186869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:21.187790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:21.188521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 24/49] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:21.191196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:21.294523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f039361-134b-4fe9-970a-d5000e5623cc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:21.330769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:21.331441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.331725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.331998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.333149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:21.333462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:21.334924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:21.344703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 26/50] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:21.350232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:21.404946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:21.456203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40973a46-6e4b-477b-8419-d166de3f7a44 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:21.491090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d7597-e27e-4793-911a-aebbb481511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:21.491979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7597-e27e-4793-911a-aebbb481511b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7597-e27e-4793-911a-aebbb481511b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.492518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7597-e27e-4793-911a-aebbb481511b None admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.493743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7597-e27e-4793-911a-aebbb481511b None admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:21.495290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7597-e27e-4793-911a-aebbb481511b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:21.496928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 25/51] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:21 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:21.503099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:21.530293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:21.601724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097d1b5-add6-49ad-bea7-0a15fc427653 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.601898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097d1b5-add6-49ad-bea7-0a15fc427653 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.602204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097d1b5-add6-49ad-bea7-0a15fc427653 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.603783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 26/52] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:21.609295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.609586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.609865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.610514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2690f1-7704-4667-8fc7-e8e9aad3e404 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:21.629399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:21.631161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:21.652329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:21.653135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.653416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.653688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.655466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:21.657016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:21.658643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 27/53] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:21.664705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:21.781320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:21.791730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeff78-3bb9-4059-bce3-1e154c79e357 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:21.816955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 27/54] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:21.828139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:21.828900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:21.829252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:21.829600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:21.830513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.domain.id=default, user.id=68ba5f09c78c41f18a7e1c6789016a58) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:21.830944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:21.832472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:21.838587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 28/55] 10.222.0.64 () {56 vars in 1322 bytes} [Fri Jun 24 10:02:21 2022] GET /identity/v3/role_assignments?user.id=68ba5f09c78c41f18a7e1c6789016a58&scope.domain.id=default&role.id=b10ab4271e1845e4aefab70d82a12def => generated 530 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:21.840503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:21.898459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:21.956990 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a4fb-d00a-417a-b131-46b1310fd5fb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:22.002076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de97aa7-7a4d-401a-9889-3c037926f781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:22.003759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de97aa7-7a4d-401a-9889-3c037926f781 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:22.004006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de97aa7-7a4d-401a-9889-3c037926f781 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:22.004272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de97aa7-7a4d-401a-9889-3c037926f781 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:22.005538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de97aa7-7a4d-401a-9889-3c037926f781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:22.007880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de97aa7-7a4d-401a-9889-3c037926f781 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:22.009567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 28/56] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:22.015524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:22.118464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18f37-e7eb-4dcc-9feb-925df1129224 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:22.156879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3cd2a-2cf6-4fa8-b68f-170d93e2fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:22.157757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3cd2a-2cf6-4fa8-b68f-170d93e2fa08 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:22.158196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3cd2a-2cf6-4fa8-b68f-170d93e2fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:22.158625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3cd2a-2cf6-4fa8-b68f-170d93e2fa08 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:22.159820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3cd2a-2cf6-4fa8-b68f-170d93e2fa08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:22.162166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3cd2a-2cf6-4fa8-b68f-170d93e2fa08 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:22.195478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 29/57] 10.222.0.64 () {60 vars in 1140 bytes} [Fri Jun 24 10:02:22 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:24.249599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cfbc7-5c9f-48c2-b28c-71ab76f5615e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:24.250424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cfbc7-5c9f-48c2-b28c-71ab76f5615e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:24.250564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cfbc7-5c9f-48c2-b28c-71ab76f5615e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:24.252723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 29/58] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:24.261434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:24.261885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:24.261997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:24.285756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:24.287000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:24.288997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:24.360319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 30/59] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:24.386103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cfbc7-5c9f-48c2-b28c-71ab76f5615e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:24.506981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cfbc7-5c9f-48c2-b28c-71ab76f5615e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:24.523707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:24.542205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4800a-76a1-4f59-ab78-604de7fc26a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:24.543057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4800a-76a1-4f59-ab78-604de7fc26a3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:24.543346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4800a-76a1-4f59-ab78-604de7fc26a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:24.543627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4800a-76a1-4f59-ab78-604de7fc26a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:24.544820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4800a-76a1-4f59-ab78-604de7fc26a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:24.546545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4800a-76a1-4f59-ab78-604de7fc26a3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:24.548235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 30/60] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:24.553918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:24.656965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50e6ed-7773-4bc9-8afa-90dc9b0ba153 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:24.694494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52accc-8850-49fd-b101-ba50d72eaefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:24.695487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52accc-8850-49fd-b101-ba50d72eaefb None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:24.695851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52accc-8850-49fd-b101-ba50d72eaefb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:24.696237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52accc-8850-49fd-b101-ba50d72eaefb None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:24.697368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52accc-8850-49fd-b101-ba50d72eaefb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:24.699605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f52accc-8850-49fd-b101-ba50d72eaefb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:24.713915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:24.727875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 31/61] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:24 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:27.938908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa126ae-ac7f-45ac-868f-9cef26458a15 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:27.939454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa126ae-ac7f-45ac-868f-9cef26458a15 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:27.939454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa126ae-ac7f-45ac-868f-9cef26458a15 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:27.940874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 31/62] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:27.946212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:27.947024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:27.947447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:27.971838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:27.973294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:28.038208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c77af-293c-4689-ac2c-2b2b144f9b73 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.038758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c77af-293c-4689-ac2c-2b2b144f9b73 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.039183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c77af-293c-4689-ac2c-2b2b144f9b73 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.040734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 32/63] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:28.046975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.047473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.047926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.072872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:28.077167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:28.099555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 32/64] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:28.147609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.202686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 33/65] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:28.231435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.295642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f31427-0d1b-4d12-8181-1ba6f4fbdb4f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:28.328333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:28.348811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36f96d-c8a4-4188-8b38-fa439e500e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:28.349653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36f96d-c8a4-4188-8b38-fa439e500e4c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.350104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36f96d-c8a4-4188-8b38-fa439e500e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.350526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36f96d-c8a4-4188-8b38-fa439e500e4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.366931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36f96d-c8a4-4188-8b38-fa439e500e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2a36f96d-c8a4-4188-8b38-fa439e500e4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:28.376010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:28.376980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:28.377775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:28.378390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 33/66] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:28.379795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006221bb-27e7-448e-97ad-5d4809671ad1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.380149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006221bb-27e7-448e-97ad-5d4809671ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.380492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006221bb-27e7-448e-97ad-5d4809671ad1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.382253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 34/67] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:28.390406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.390728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.391010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.397209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d195-b841-40de-95c4-ffff2b273c24 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:28.413920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:28.416000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:28.417523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:28.442267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:28.443318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.443708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.444093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.459989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:28.474412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:28.475380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:28.476415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 34/68] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:28.477026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.507082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 35/69] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:28.508342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.594008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c172c-5cc8-4c0e-8221-9db9fcddfe70 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:28.642218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:28.642953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.643258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.643539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.644411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:28.644707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:28.646369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef41bdd-2ad7-42b8-8b39-949a4f3fad4d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:28.655272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a214420-3066-4b60-ad3e-a8ea765823c1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:28.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 35/70] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:28.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d280f51-db4c-4521-a051-2431c7ae5b42 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d280f51-db4c-4521-a051-2431c7ae5b42 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d280f51-db4c-4521-a051-2431c7ae5b42 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 36/71] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:28.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d280f51-db4c-4521-a051-2431c7ae5b42 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.680108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:28.709166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:28.712950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:28.713641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.713924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.714197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.715446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:28.715761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:28.717751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:28.731826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 36/72] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:28.731826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.785774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d280f51-db4c-4521-a051-2431c7ae5b42 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:28.829043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2734-90fb-42c3-ada1-a7417f224fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:28.829891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2734-90fb-42c3-ada1-a7417f224fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.830179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2734-90fb-42c3-ada1-a7417f224fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.830467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2734-90fb-42c3-ada1-a7417f224fb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.841275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf2734-90fb-42c3-ada1-a7417f224fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:28.843249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4024a-bf36-4886-8725-a3d18783dcec None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-dedf2734-90fb-42c3-ada1-a7417f224fb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:28.849117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:28.849967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:28.850746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:28.854858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 37/73] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:28.856691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7984d6-2921-4ad8-a1be-45d2ac7d11cc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.857021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7984d6-2921-4ad8-a1be-45d2ac7d11cc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.857673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7984d6-2921-4ad8-a1be-45d2ac7d11cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.874737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6a7984d6-2921-4ad8-a1be-45d2ac7d11cc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:28.876024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6a7984d6-2921-4ad8-a1be-45d2ac7d11cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:28.889562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:28.890904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.891319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.891737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.907006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:28.908488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:28.910128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 37/74] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:28.910918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.944436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:28.945244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 38/75] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:28.946603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e90c-1f00-43d1-9296-b53250f20473 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:28.947045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e90c-1f00-43d1-9296-b53250f20473 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:28.947232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e90c-1f00-43d1-9296-b53250f20473 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:28.953598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 39/76] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:28.953598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544e90c-1f00-43d1-9296-b53250f20473 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:28.977689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.016393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a06c6a-352b-4269-a8e1-78ce7a2bae00 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.050757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.051571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.051857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.052131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.053345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.054839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.056198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544e90c-1f00-43d1-9296-b53250f20473 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.061117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 38/77] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:29.061117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.091864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.092545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.092833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.093117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.093871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:29.094184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.095609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d92fc8-7e1e-4b2d-bf75-a5d15b8582ce None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.103714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 40/78] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:29.105203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.105500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.105786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.121504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:29.122746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:29.163071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b1aa2-3b34-41fc-88c4-79baec076bd3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.164734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.192059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 41/79] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:29.192926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.204013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.204678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.204960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.205237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.213131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.276151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.277600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.295433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1761bd-44c3-47ec-8aec-0d79399b88fb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.328695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.329430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.329719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.329999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.340085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:29.347765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:29.348605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:29.349465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 42/80] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:29.350082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.379255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 39/81] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:29.380100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.428989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.451326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89817a4-15c6-444f-9306-5af791bb18b5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.479156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05165-50f5-448f-bfd3-f460ea1f2df4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.485213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.485881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.486172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.486454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.515083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.516372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.516663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.516943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.518198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.519731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.521388 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 40/82] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:29.522199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.560016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.561738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.591325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.624062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651982e-5934-4f78-a601-11d715c3a926 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.641428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.657392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.658184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.658462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.658759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.659920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.661386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.663022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 41/83] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:29.663022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.667647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 43/84] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:29.667647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.766660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7df74-abb5-418f-a482-f69ae501c3ab None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.773000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18bdb6-65a4-4e3b-80ae-bcbff6ea5f9a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.805651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.806529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.806851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.807136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.808339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.808865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.809533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.809815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.809917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.810091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.810920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:29.811231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.811606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 44/85] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:29.812377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.812629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.818023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.825968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 42/86] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:29.825968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:29.866256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:29.921296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ec85e-f0b6-413e-9096-0aa2cef763ed None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.922438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e62fdd-9d78-4e46-a1cd-4a07cb99f802 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:29.954822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.955481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.955589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:29.955773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.956055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.956253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:29.956535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:29.956811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:29.957526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:29.959009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:29.979081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 43/87] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:29 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:29.987789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.034403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.035970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.054154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:30.088328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:30.112861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f1211-0c1b-48cf-997a-02bcd416b2cb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.150335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 45/88] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:30.151808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.161925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.162982 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.163196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.163565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.164652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.166142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.167918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 44/89] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:29 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:30.168689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.279653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b1da0-4dc4-44de-9963-769437f5d83b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.301889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab476dc-3a7a-42c8-88d2-2bc3d4bee2ac None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.321645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.322421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.322821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.323106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.323952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=e9d9ed1ce844447183749518f6265355) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:30.324267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.325737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.331714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:30.338927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 46/90] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:30 2022] GET /identity/v3/role_assignments?user.id=e9d9ed1ce844447183749518f6265355&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:30.339937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.346908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.348140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.349639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.350153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.351763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.359194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.361361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 45/91] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:30 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:30.363124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.372238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:30.547421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f727-e35d-4527-bce7-47a5d035afdb None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.559650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd606f4-3051-4830-88f1-35c2f8f82e16 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.590729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.590729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.590729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.590729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.591138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.591138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.591138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 47/92] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:30 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:30.591864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.599250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.599921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.600206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.600483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.601338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] RBAC: Adding query filter params (role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=e9d9ed1ce844447183749518f6265355) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:30.601651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.603168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.612934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 46/93] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:30 2022] GET /identity/v3/role_assignments?user.id=e9d9ed1ce844447183749518f6265355&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:30.613789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:30.727108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:30.749707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:30.793894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73badc5-4cad-4a6a-9481-cc00850f9bbe None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.823690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42effc85-3305-4668-8b91-d373c7c35d78 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:30.834234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.835493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.835783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.836063 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.884414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:30.885203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:30.885488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:30.885766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:30.886751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=68ba5f09c78c41f18a7e1c6789016a58) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:30.887072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.888563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:30.901457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 47/94] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:30 2022] GET /identity/v3/role_assignments?user.id=68ba5f09c78c41f18a7e1c6789016a58&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:30.914114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:30.915709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:31.020128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:31.047395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 48/95] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:31.053135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:31.188027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa73ed-487a-4ce0-9f46-1a1666698d04 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:31.211109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:31.234941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9c9de-8c93-4960-b667-16bf73e71d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:31.235849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9c9de-8c93-4960-b667-16bf73e71d76 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:31.236137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9c9de-8c93-4960-b667-16bf73e71d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:31.236414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9c9de-8c93-4960-b667-16bf73e71d76 None admin] PATH_INFO: `/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:31.237682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9c9de-8c93-4960-b667-16bf73e71d76 None admin] RBAC: Authorizing `identity:get_project(project_id=778ab12e1e684f43aebaf63e47b6d6cf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:31.239389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9c9de-8c93-4960-b667-16bf73e71d76 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:31.241194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 48/96] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:31 2022] GET /identity/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:31.247158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:31.397507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eefa24-3903-4eeb-9682-1060dee0ac89 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:31.447532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:31.448130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:31.448422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:31.448706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:31.449569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=778ab12e1e684f43aebaf63e47b6d6cf, user.id=e9d9ed1ce844447183749518f6265355) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:31.449887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:31.451490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8560c5-33f4-4256-8649-d99a9d3d5f55 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:31.457633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 49/97] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:31 2022] GET /identity/v3/role_assignments?user.id=e9d9ed1ce844447183749518f6265355&scope.project.id=778ab12e1e684f43aebaf63e47b6d6cf&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:32.793703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c40863-bde9-4832-b200-469004d5227b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:32.794217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c40863-bde9-4832-b200-469004d5227b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:32.794288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c40863-bde9-4832-b200-469004d5227b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:32.795784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 49/98] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:32.812174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377715d-3103-422e-b790-cc3848904635 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:32.812615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377715d-3103-422e-b790-cc3848904635 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:32.812802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377715d-3103-422e-b790-cc3848904635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:32.835699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9377715d-3103-422e-b790-cc3848904635 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:32.836912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9377715d-3103-422e-b790-cc3848904635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:32.884768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:32.930297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 50/99] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:32.967224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c40863-bde9-4832-b200-469004d5227b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:33.092602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c40863-bde9-4832-b200-469004d5227b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:33.137801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a932414-edd3-4804-a0ce-8d565b7cc641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:33.138664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a932414-edd3-4804-a0ce-8d565b7cc641 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.138975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a932414-edd3-4804-a0ce-8d565b7cc641 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.139255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a932414-edd3-4804-a0ce-8d565b7cc641 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.140559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a932414-edd3-4804-a0ce-8d565b7cc641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:33.142063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a932414-edd3-4804-a0ce-8d565b7cc641 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:33.143924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 50/100] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:33.149353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377715d-3103-422e-b790-cc3848904635 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:33.185976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9472e44-b002-4545-98d8-b784d71cfdf6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.186322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9472e44-b002-4545-98d8-b784d71cfdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.186545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9472e44-b002-4545-98d8-b784d71cfdf6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.188129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 51/101] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:33.193287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30078bee-47d8-4b67-b24e-be885b207fd1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.193575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30078bee-47d8-4b67-b24e-be885b207fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.193854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30078bee-47d8-4b67-b24e-be885b207fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.224700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-30078bee-47d8-4b67-b24e-be885b207fd1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:33.225893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-30078bee-47d8-4b67-b24e-be885b207fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:33.295429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:33.313880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 52/102] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:33.315320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b99ec-68ef-45a0-b1ba-8c70dac0bad4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.316018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b99ec-68ef-45a0-b1ba-8c70dac0bad4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.316481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b99ec-68ef-45a0-b1ba-8c70dac0bad4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.317765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 53/103] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:33.323689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.323981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.324264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.342241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:33.343456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:33.372717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377715d-3103-422e-b790-cc3848904635 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:33.416603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:33.417280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.417564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.417842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.418585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:33.420198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:33.442066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 54/104] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:33.442970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:33.468707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 51/105] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:02:33 2022] POST /identity/v3/projects => generated 302 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:33.485991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:33.538962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:33.663155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65768ca7-157d-494b-864c-c2c76fc15c58 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:33.689926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a7e85-4b7c-4c75-9060-8a556b2caf7b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:33.699984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:33.735528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:33.736419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.736637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.736918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.751193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:33.759129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:33.760157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:33.760598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:33.761015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:33.765441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:33.766498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:33.767492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:33.769460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 55/106] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:33.775369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:33.787385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:33.796883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:33.797963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:33.799034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 52/107] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:02:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:02:33.804170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:33.960638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616df32d-9e4d-4c44-9b0d-e0a799ea30d8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:33.974342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.001838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b766429-961a-41f7-928a-05770b6191e2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.024348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.025166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.025471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.025753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.026524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:34.026867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.028302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29d219-ae3c-4bc1-8140-f45b037a1f47 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.042400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 56/108] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:34.044394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5c85e-267a-440b-9774-13a47b56c6c4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.044691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5c85e-267a-440b-9774-13a47b56c6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.045001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5c85e-267a-440b-9774-13a47b56c6c4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.046315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 57/109] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:34.047607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5c85e-267a-440b-9774-13a47b56c6c4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.067852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.068834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.069126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.069408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.070175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:34.070488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.077285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463d7f1-b5c4-4dc4-8e47-1adde51f3c99 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.093746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 53/110] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:02:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:34.097135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.097431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.097726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.123138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.134201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:34.137235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:34.231723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5c85e-267a-440b-9774-13a47b56c6c4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.257880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 54/111] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:34.258692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.290910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f40a18-56e4-4ee7-8f98-e90642da90e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.291859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f40a18-56e4-4ee7-8f98-e90642da90e1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.292144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f40a18-56e4-4ee7-8f98-e90642da90e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.292564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f40a18-56e4-4ee7-8f98-e90642da90e1 None admin] PATH_INFO: `/v3/users/6daf0a65684d416081460eac61880c28` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.307366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f40a18-56e4-4ee7-8f98-e90642da90e1 None admin] RBAC: Authorizing `identity:get_user(user_id=6daf0a65684d416081460eac61880c28)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.308875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f40a18-56e4-4ee7-8f98-e90642da90e1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.311004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 58/112] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/users/6daf0a65684d416081460eac61880c28 => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:34.312413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981fa26-9cca-44c5-9bcc-3b295bb83afb None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.312706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981fa26-9cca-44c5-9bcc-3b295bb83afb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.312986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981fa26-9cca-44c5-9bcc-3b295bb83afb None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.314242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 59/113] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:34.317524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981fa26-9cca-44c5-9bcc-3b295bb83afb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.348800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.377910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7306b-c9a1-4d24-901f-0fa614cc508f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.415438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c4153-7bb7-47d1-b1d1-b1a068646b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.416453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c4153-7bb7-47d1-b1d1-b1a068646b7a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.416870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c4153-7bb7-47d1-b1d1-b1a068646b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.417019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c4153-7bb7-47d1-b1d1-b1a068646b7a None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.417980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.418244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c4153-7bb7-47d1-b1d1-b1a068646b7a None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.420099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c4153-7bb7-47d1-b1d1-b1a068646b7a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.428234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 55/114] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:34.428234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7a1e8-46b5-409d-9679-99e766f3be92 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.428234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7a1e8-46b5-409d-9679-99e766f3be92 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.428234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7a1e8-46b5-409d-9679-99e766f3be92 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.428234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 56/115] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:34.428234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7a1e8-46b5-409d-9679-99e766f3be92 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.432140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981fa26-9cca-44c5-9bcc-3b295bb83afb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.470388 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea844351-5413-4787-afc6-da8e0efc57e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.471272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844351-5413-4787-afc6-da8e0efc57e5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.471558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844351-5413-4787-afc6-da8e0efc57e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.471833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844351-5413-4787-afc6-da8e0efc57e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.483414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea844351-5413-4787-afc6-da8e0efc57e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ea844351-5413-4787-afc6-da8e0efc57e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:34.491405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:34.492259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:34.493020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 60/116] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:34.494253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.494253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.494472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.512243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:34.513351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:34.549234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7a1e8-46b5-409d-9679-99e766f3be92 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.585549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 61/117] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:34.585549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.593884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.593884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.593884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.593884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.613442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.677956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.679499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.681381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.697997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea363a-dda4-40f7-9174-b68fb0edd3bd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.739971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7903575-868b-48fb-a6fe-7d1fa8c45912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.741346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903575-868b-48fb-a6fe-7d1fa8c45912 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.741644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903575-868b-48fb-a6fe-7d1fa8c45912 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.742176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7903575-868b-48fb-a6fe-7d1fa8c45912 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.743672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7903575-868b-48fb-a6fe-7d1fa8c45912 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.745816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7903575-868b-48fb-a6fe-7d1fa8c45912 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.752931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 62/118] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:34.752931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.752931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.752931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.772069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:34.773174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:34.801798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 57/119] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:34.806448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.854771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 63/120] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:34.854771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:34.863181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.920714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996327c7-d114-46c6-a103-b52330b54bec None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.949121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:34.966035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170665f2-f91e-43ec-8ae7-ff8d3ca4d1aa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:34.970744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:34.971487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:34.971780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:34.972057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:34.972844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:34.973158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:34.974618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:34.992460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 58/121] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:34.992460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.003883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.004693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.004979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.005233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.016023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:35.023733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:35.024801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 64/122] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:35.026615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.124415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a5970-5090-4c0a-a83c-6b3bdb3f1c02 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.130272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc699b-2ace-468d-b22f-498f67a15358 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.132239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:35.160710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.161639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.161901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.162177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/e9d9ed1ce844447183749518f6265355/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.164436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=536792d0280e402db36f3fcc605a2dbd, user_id=e9d9ed1ce844447183749518f6265355, role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.165899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.167685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.168561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.169057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.169166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.170697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.172264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.178866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 65/123] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.178866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.184837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 59/124] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:34 2022] PUT /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/e9d9ed1ce844447183749518f6265355/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:35.190856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.198549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:35.313936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ca7d5-3cc2-4c82-ae4e-28e29f29d797 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.327662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f87ee-aaaa-43df-a3b6-81ce028646b5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.357276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.358399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.359053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.359472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.370838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.371024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:35.371752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.372033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.372350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.373826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.375542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.377519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 60/125] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.378122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:35.379364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:35.380242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:35.381011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 66/126] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:35.381683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.418567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:35.467390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:35.494445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84c56-93d8-493f-b642-b235fbccca13 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.504963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7eaa23-286d-4320-bc05-ced49928cc03 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.539087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.540852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.541300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.541999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.543055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:35.543606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.545630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.546355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.547250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.547781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.548111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.549196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] RBAC: Adding query filter params (role.id=a23a16fa358b491c876189b3d05e60c7, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=6daf0a65684d416081460eac61880c28) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:35.549569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.551320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.557281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 61/127] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.558300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.558421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 67/128] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/role_assignments?user.id=6daf0a65684d416081460eac61880c28&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=a23a16fa358b491c876189b3d05e60c7 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.559559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.679665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:35.684709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7d20-bb5b-4a5c-a64c-049c51b01479 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.728343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.729248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.729908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.730192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.731508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.733201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.736279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 62/129] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.737358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.766172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e7e9-a49a-49e4-ad73-1dd005f7745c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.809075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.809838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.810123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.810398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.811219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:35.811527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.813129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.823830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:35.837641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 68/130] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.838486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.860620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b138f0-5ba7-4de1-ad9c-0969a44e5ccd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:35.902196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:35.902196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:35.902196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:35.902196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:35.903533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:35.905260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:35.909809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 63/131] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:35.909809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:35.944896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:36.023906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9588ff6-a62b-4e5c-ab0a-a28ca4da316b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.037905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeac7b-bb4c-4447-b4ed-d3af43eb4ee8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.062090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.063412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.063708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.063993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.065230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.066736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.068849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 64/132] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:35 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:36.069636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:36.110272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.111298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.111590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.111867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/e9d9ed1ce844447183749518f6265355/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.117725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] RBAC: Authorizing `identity:create_grant(project_id=536792d0280e402db36f3fcc605a2dbd, user_id=e9d9ed1ce844447183749518f6265355, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.119416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.156158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 69/133] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:35 2022] PUT /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/e9d9ed1ce844447183749518f6265355/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:36.157114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:36.180020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:36.184518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c66162-5fff-4a91-bace-35bb35116129 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.267879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:36.273078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8ee72-9c56-4f3c-9445-b1db0d551577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.274034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8ee72-9c56-4f3c-9445-b1db0d551577 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.274320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8ee72-9c56-4f3c-9445-b1db0d551577 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.274597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8ee72-9c56-4f3c-9445-b1db0d551577 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.279587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8ee72-9c56-4f3c-9445-b1db0d551577 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.281874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8ee72-9c56-4f3c-9445-b1db0d551577 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.284267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 65/134] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:36 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:36.285617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd59e-b8c0-44a6-8c42-b884d2a47811 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.285909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd59e-b8c0-44a6-8c42-b884d2a47811 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.286244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd59e-b8c0-44a6-8c42-b884d2a47811 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.287694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 66/135] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:36.292704 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd59e-b8c0-44a6-8c42-b884d2a47811 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:36.338541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2be4f6-ed2e-4d62-ae19-0fe66355730d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.397752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5c541-c60e-4215-8fa0-ab07f427ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.398673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5c541-c60e-4215-8fa0-ab07f427ad57 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.398985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5c541-c60e-4215-8fa0-ab07f427ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.399271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5c541-c60e-4215-8fa0-ab07f427ad57 None admin] PATH_INFO: `/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.401163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5c541-c60e-4215-8fa0-ab07f427ad57 None admin] RBAC: Authorizing `identity:get_project(project_id=778ab12e1e684f43aebaf63e47b6d6cf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.402933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5c541-c60e-4215-8fa0-ab07f427ad57 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.405860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 70/136] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:36 2022] GET /identity/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:36.407156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.407448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.407731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.427781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:36.429745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:36.444267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd59e-b8c0-44a6-8c42-b884d2a47811 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.495309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.496281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.496562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.496838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/68ba5f09c78c41f18a7e1c6789016a58/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.501585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] RBAC: Authorizing `identity:create_grant(project_id=536792d0280e402db36f3fcc605a2dbd, user_id=68ba5f09c78c41f18a7e1c6789016a58, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.503132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.519233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 71/137] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:36.520169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:36.530439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 67/138] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:36 2022] PUT /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/68ba5f09c78c41f18a7e1c6789016a58/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:36.562328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:36.615054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:36.617790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:36.708825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf32374-c45b-4c3b-8914-092466197cee None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.778138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.779360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.779701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.780188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] PATH_INFO: `/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf/users/e9d9ed1ce844447183749518f6265355/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.783426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=778ab12e1e684f43aebaf63e47b6d6cf, user_id=e9d9ed1ce844447183749518f6265355, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.787475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.798304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774590c-a939-49e4-a235-539c720d5f15 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:36.816278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 72/139] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:36 2022] PUT /identity/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf/users/e9d9ed1ce844447183749518f6265355/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:36.856094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2d19-fc94-4463-ac42-c901e6b55022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:36.856924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2d19-fc94-4463-ac42-c901e6b55022 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:36.857210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2d19-fc94-4463-ac42-c901e6b55022 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:36.857605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2d19-fc94-4463-ac42-c901e6b55022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:36.859867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2d19-fc94-4463-ac42-c901e6b55022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:36.861459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c2d19-fc94-4463-ac42-c901e6b55022 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:36.863585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 68/140] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:36.869512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:37.001713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc721e9-96bd-45b4-ba47-71d0977798b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:37.027150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:37.048775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855c3c9-0766-4122-84e9-c86a62dfcc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:37.049548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855c3c9-0766-4122-84e9-c86a62dfcc4c None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:37.049645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855c3c9-0766-4122-84e9-c86a62dfcc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:37.049884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855c3c9-0766-4122-84e9-c86a62dfcc4c None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:37.050643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855c3c9-0766-4122-84e9-c86a62dfcc4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:37.052192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855c3c9-0766-4122-84e9-c86a62dfcc4c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:37.084822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 73/141] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:36 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:37.977295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd654f29-8448-4e5f-a09b-72fb81a2eb1b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:37.977795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd654f29-8448-4e5f-a09b-72fb81a2eb1b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:37.977795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd654f29-8448-4e5f-a09b-72fb81a2eb1b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:37.979226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 69/142] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:37.985147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:37.985450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:37.985730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:38.003143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:38.004381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:38.078761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 74/143] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:38.102224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd654f29-8448-4e5f-a09b-72fb81a2eb1b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:38.214022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:38.222716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd654f29-8448-4e5f-a09b-72fb81a2eb1b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:38.260625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:38.261376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:38.261663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:38.261949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:38.272640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:38.284062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:38.284927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:38.285674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 70/144] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:02:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:38.291208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:38.417318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab672-9333-4cb2-94fe-bcfb035c8c26 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:38.418963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:38.458883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:38.459568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:38.459909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:38.460190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:38.461033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:38.461341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:38.462786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:38.474780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 75/145] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:02:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:38.480732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:38.603597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a016769-5c08-417e-aa20-63b64c9e39f8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:38.642993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681f2cf-e1e9-4767-846f-723c5d63cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:38.643903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681f2cf-e1e9-4767-846f-723c5d63cc69 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:38.644192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681f2cf-e1e9-4767-846f-723c5d63cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:38.644472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681f2cf-e1e9-4767-846f-723c5d63cc69 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:38.645837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681f2cf-e1e9-4767-846f-723c5d63cc69 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:38.647452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681f2cf-e1e9-4767-846f-723c5d63cc69 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:38.649249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 71/146] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:38 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:38.655083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:38.790397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950696-a94e-4508-a89a-724ea759a69d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:38.832771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e122234-bca0-4761-8d24-25e92765a47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:38.833436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e122234-bca0-4761-8d24-25e92765a47c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:38.833720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e122234-bca0-4761-8d24-25e92765a47c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:38.833999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e122234-bca0-4761-8d24-25e92765a47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:38.874801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:38.913856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e122234-bca0-4761-8d24-25e92765a47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:38.915382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e122234-bca0-4761-8d24-25e92765a47c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:38.966629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951975de-6d0d-4af1-be8e-dbc5f16aaa6f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:38.967036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951975de-6d0d-4af1-be8e-dbc5f16aaa6f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:38.967298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951975de-6d0d-4af1-be8e-dbc5f16aaa6f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:38.968807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 72/147] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:38.976082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc71d-7099-4438-ba72-0da6fdd772a4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:38.976377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc71d-7099-4438-ba72-0da6fdd772a4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:38.976665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc71d-7099-4438-ba72-0da6fdd772a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.007908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-925fc71d-7099-4438-ba72-0da6fdd772a4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:39.010089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-925fc71d-7099-4438-ba72-0da6fdd772a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:39.059082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:39.068439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 76/148] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:39.070034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea840e86-9aaf-4e8b-a711-01569951a282 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.070486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea840e86-9aaf-4e8b-a711-01569951a282 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.074953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea840e86-9aaf-4e8b-a711-01569951a282 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.076451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 77/149] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:39.078134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea840e86-9aaf-4e8b-a711-01569951a282 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.134462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 73/150] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:39.136989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.137916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.139746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.169203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:39.173960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:39.192708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:39.251463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea840e86-9aaf-4e8b-a711-01569951a282 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:39.287383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 74/151] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:39.288071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.311226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:39.312163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.312445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.312723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.314865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:39.316492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:39.318781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 78/152] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:39.324600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.381825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:39.412769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a967e5-f07b-474d-bed8-90b53bf778ea None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:39.490486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b83ac-1ed6-4d1e-8560-5ef93fcb73bd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:39.502977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:39.503829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.504120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.504408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.527651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:39.535521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:39.536387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:39.537156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 75/153] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:39.537789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.568584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:39.584315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:39.584992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.585274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.585543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.586395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] RBAC: Adding query filter params (role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=e9d9ed1ce844447183749518f6265355) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:39.586725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:39.588227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:39.605970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 79/154] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/role_assignments?user.id=e9d9ed1ce844447183749518f6265355&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:39.606939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.774589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2e71f-38e0-4d92-8a55-23b8e7eabdee None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:39.811768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c1245-abe6-43a1-9018-9df5db54d828 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:39.845869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:39.847130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.847633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.848151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.868666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:39.869370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:39.869649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:39.869922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:39.870690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:39.871023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:39.872477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:39.879507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:39.886597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 80/155] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:39.894741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:39.895831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:39.896674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:39.897250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.899646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 76/156] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:39.906206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:39.919108 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:39.995181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:40.048805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feecc90-da79-461e-91b4-c0f49d363e71 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.088027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.088923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.089207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.089482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.090725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.092351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.093199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833a90-9d4b-4364-8ea4-3d232a338020 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.093994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 81/157] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:40.099555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:40.146643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.147372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.147658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.147939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.148708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:40.149020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.150428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.165706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 77/158] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:40.171614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:40.213034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b80d6-1f44-41ff-9558-e12c308a1b51 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.255179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:40.268090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:40.268509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67289b-2817-4a25-a417-e2cfd482d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.269287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67289b-2817-4a25-a417-e2cfd482d881 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.269666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67289b-2817-4a25-a417-e2cfd482d881 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.270040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67289b-2817-4a25-a417-e2cfd482d881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.319397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6109ce-5a82-4bba-a3d8-d50c0b42ff75 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.364276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67289b-2817-4a25-a417-e2cfd482d881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.365920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67289b-2817-4a25-a417-e2cfd482d881 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.372445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.373387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.373673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.373951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] PATH_INFO: `/v3/users/6daf0a65684d416081460eac61880c28` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.378049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] RBAC: Authorizing `identity:get_user(user_id=6daf0a65684d416081460eac61880c28)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.379639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.381749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 78/159] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:40 2022] GET /identity/v3/users/6daf0a65684d416081460eac61880c28 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:40.386989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:40.520894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 82/160] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:02:40.522226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2d784-6ae1-4e5a-87a6-551dcfb53d6d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.522620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2d784-6ae1-4e5a-87a6-551dcfb53d6d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.523020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2d784-6ae1-4e5a-87a6-551dcfb53d6d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.525479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 83/161] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:40.530523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2d784-6ae1-4e5a-87a6-551dcfb53d6d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:40.575716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:40.590670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbedcff-1764-4066-a6d6-ebbe45cd80df None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.627400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bde0ef-fd3c-49e9-ba4e-ab23389620b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.628288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bde0ef-fd3c-49e9-ba4e-ab23389620b5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.628687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bde0ef-fd3c-49e9-ba4e-ab23389620b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.628966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bde0ef-fd3c-49e9-ba4e-ab23389620b5 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.630282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bde0ef-fd3c-49e9-ba4e-ab23389620b5 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.631802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bde0ef-fd3c-49e9-ba4e-ab23389620b5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.636432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 79/162] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:40 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:40.636432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.636432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.636432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.641763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:40.654522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:40.655841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:40.725945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 80/163] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:40.726846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:40.739780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2d784-6ae1-4e5a-87a6-551dcfb53d6d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.810976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.812334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.812832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.813321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.819844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.821350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.828430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 84/164] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:40 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:40.829302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:40.876498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a5ba6-441c-4de1-b686-8a2af0858a7c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:40.908918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:40.913584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:40.914594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:40.915198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:40.915488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/6daf0a65684d416081460eac61880c28/roles/a23a16fa358b491c876189b3d05e60c7` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:40.918056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] RBAC: Authorizing `identity:create_grant(project_id=536792d0280e402db36f3fcc605a2dbd, user_id=6daf0a65684d416081460eac61880c28, role_id=a23a16fa358b491c876189b3d05e60c7)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:40.919696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:40.943766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 81/165] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:40 2022] PUT /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd/users/6daf0a65684d416081460eac61880c28/roles/a23a16fa358b491c876189b3d05e60c7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:40.944645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:41.107181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:41.116329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d9c0-83e6-445d-a8f5-7259ad7f9948 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:41.124935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc8ebc-d728-46d9-ba4f-6e7c844dfcbd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:41.161790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:41.162573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.163021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.163300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.164244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=e9d9ed1ce844447183749518f6265355) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:41.164568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:41.166230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f1998-e075-4828-84a5-7f685f83122b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:41.172797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 82/166] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:40 2022] GET /identity/v3/role_assignments?user.id=e9d9ed1ce844447183749518f6265355&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:41.180065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:41.180251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4b87d-f785-4ff2-9c9d-98c940d2c126 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.180540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4b87d-f785-4ff2-9c9d-98c940d2c126 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.180822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4b87d-f785-4ff2-9c9d-98c940d2c126 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.180936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.181191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.181415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.182542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 83/167] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:41.196633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.196962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.197190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.204410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:41.214291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:41.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:41.221206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:41.224066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 85/168] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:41.230754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:41.240253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:41.241490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:41.367171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:41.405402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 84/169] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:41.418117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad667f1c-ea2d-4ac1-bafc-30bb37b06b68 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:41.449054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:41.469870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:41.470569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.470870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.471150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.471983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:41.472293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:41.474104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:41.492126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 86/170] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:41.498219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:41.548684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:41.648406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3053c6-d579-4417-9b2d-12a6d1e6d0e8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:41.717129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d7307-6286-4aee-8f60-ef23a1e58289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:41.718358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d7307-6286-4aee-8f60-ef23a1e58289 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.718815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d7307-6286-4aee-8f60-ef23a1e58289 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.719235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d7307-6286-4aee-8f60-ef23a1e58289 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.720662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21513bbc-efc0-453c-a010-be08fefb4c4c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:41.732966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d7307-6286-4aee-8f60-ef23a1e58289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-533d7307-6286-4aee-8f60-ef23a1e58289 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:41.742160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:41.743170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:41.744068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:41.744657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 85/171] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:02:41.745498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ef4b-b9ae-485c-a8f2-80200c13af9b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.745679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ef4b-b9ae-485c-a8f2-80200c13af9b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.745968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ef4b-b9ae-485c-a8f2-80200c13af9b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.750923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 86/172] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:41.755845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.756136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.756416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.785441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:41.786668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:41.794382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:41.795399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:41.795676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:41.795963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] PATH_INFO: `/v3/users/68ba5f09c78c41f18a7e1c6789016a58` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:41.797199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=68ba5f09c78c41f18a7e1c6789016a58)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:41.798679 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:41.800543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 87/173] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:41 2022] GET /identity/v3/users/68ba5f09c78c41f18a7e1c6789016a58 => generated 247 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:41.801142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:41.866323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:41.887465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 87/174] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:41.888522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:41.997693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32dc21-f11e-4163-abe8-b537b6a95080 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.033078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5b6e1-e082-4e29-a144-c2245027a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.033754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5b6e1-e082-4e29-a144-c2245027a490 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.034044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5b6e1-e082-4e29-a144-c2245027a490 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.034529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5b6e1-e082-4e29-a144-c2245027a490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.087087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.107311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5b6e1-e082-4e29-a144-c2245027a490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.109126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5b6e1-e082-4e29-a144-c2245027a490 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.128968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e70b3-03b7-4785-a690-478984f43ed2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.132269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.164819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.165840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.166537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.167244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.168199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:42.168699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.171112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.180667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 88/175] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:42.181787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.213529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 88/176] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:42.215478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979cdf-9ebb-4c51-982c-91fd9c7a9f9d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.215774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979cdf-9ebb-4c51-982c-91fd9c7a9f9d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.216051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979cdf-9ebb-4c51-982c-91fd9c7a9f9d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.217859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 89/177] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:42.218886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1d63-6da3-4cdf-a86c-7fd724d56481 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.219336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1d63-6da3-4cdf-a86c-7fd724d56481 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.219622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1d63-6da3-4cdf-a86c-7fd724d56481 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.221121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 90/178] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:42.221846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1d63-6da3-4cdf-a86c-7fd724d56481 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.294858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923bdd9-fb24-468d-8dc1-8473f6bab944 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.302152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.329900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.330097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1d63-6da3-4cdf-a86c-7fd724d56481 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.330721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.331007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.331272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.343797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:42.351389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:42.352252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:42.353051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 89/179] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:42.355662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.366791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.367797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d54465-3309-4a79-9a4f-6f033d5d2a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.368780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d54465-3309-4a79-9a4f-6f033d5d2a5b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.369053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d54465-3309-4a79-9a4f-6f033d5d2a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.369347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d54465-3309-4a79-9a4f-6f033d5d2a5b None admin] PATH_INFO: `/v3/users/e9d9ed1ce844447183749518f6265355` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.370622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d54465-3309-4a79-9a4f-6f033d5d2a5b None admin] RBAC: Authorizing `identity:get_user(user_id=e9d9ed1ce844447183749518f6265355)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.372239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d54465-3309-4a79-9a4f-6f033d5d2a5b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.378835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 91/180] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/users/e9d9ed1ce844447183749518f6265355 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:42.378835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8546c7-2113-4e48-9ab3-a1f35b14c1a7 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.378835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8546c7-2113-4e48-9ab3-a1f35b14c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.378835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8546c7-2113-4e48-9ab3-a1f35b14c1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.394335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bb8546c7-2113-4e48-9ab3-a1f35b14c1a7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:42.395569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bb8546c7-2113-4e48-9ab3-a1f35b14c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:42.466989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 92/181] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:42.468357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.468646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.468922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.482141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4db9f-20f4-4747-a430-6a26f15baf89 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.487203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:42.488404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:42.527534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.528778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.529336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.530292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.531736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.534296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.536779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 90/182] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:42.538593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.558792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 93/183] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:42.558792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.562349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.628534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.655815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c118d-5fd1-42ef-a6ad-c3420d76687e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.674168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff0755-7990-4721-8edc-e134ae0ae801 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.694639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.695552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.695660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.695860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.696609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:42.696910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.698311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.709958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.710625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.710939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.711219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.717826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 91/184] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:42.717826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.783617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.785102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.788719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.820993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2fb3-f24c-4fbf-b1d9-b8860a1c7efc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:42.856613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:42.857642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:42.858114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:42.858199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:42.859411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:42.860891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:42.862687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 92/185] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:42.863317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.870973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:42.902228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 94/186] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:42.903102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:42.987984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b9b4c-0ce5-470c-8d17-261978fd1887 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.020045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6192-efe8-44dc-bd6a-f8e554749f0c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.023256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:43.024738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.025629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.025995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.026280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.027156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=68ba5f09c78c41f18a7e1c6789016a58) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:43.027471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.028946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.035364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 93/187] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/role_assignments?user.id=68ba5f09c78c41f18a7e1c6789016a58&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:43.036183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.073571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.074475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.074779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.075055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.091740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:43.101613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:43.102616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:43.103424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 95/188] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:02:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:43.104060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.128570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:43.168143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef32c2-e90e-437a-94e8-f3b59f4d97b3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.206360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.207628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.208088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.208568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] PATH_INFO: `/v3/users/17183b5e96664c19ac4edd822cdf8561` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.224412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] RBAC: Authorizing `identity:get_user(user_id=17183b5e96664c19ac4edd822cdf8561)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.226204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.228194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 94/189] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/users/17183b5e96664c19ac4edd822cdf8561 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:43.229181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.241010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbfe28-af6f-4990-86af-3121e43d88bb None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.298015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.299183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.299693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.299973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.301262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.302797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.337121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 96/190] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:43 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:43.341500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.358838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9161a66-ec0e-4536-8247-d8ac402c9f11 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.381925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:43.391708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:43.403035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.403977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.404387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.404853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] PATH_INFO: `/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.407313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] RBAC: Authorizing `identity:get_project(project_id=778ab12e1e684f43aebaf63e47b6d6cf)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.409763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.414337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 95/191] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/projects/778ab12e1e684f43aebaf63e47b6d6cf => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:43.415492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.509229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea99bf-a544-4920-874e-8db1b3e79d5c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.564637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.565337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.565624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.565903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.567189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:43.567508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.569401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.578927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6986a-858d-4245-a24e-803b5c3751f8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.582876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 97/192] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:43.587892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.623397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6e42e-9c93-41e1-a0d8-26fd19c80d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.625133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6e42e-9c93-41e1-a0d8-26fd19c80d02 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.625730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6e42e-9c93-41e1-a0d8-26fd19c80d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.626184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6e42e-9c93-41e1-a0d8-26fd19c80d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.697476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:43.713509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:43.723105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6e42e-9c93-41e1-a0d8-26fd19c80d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.724362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53758417-9efe-4e8b-9d65-3a788e640767 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.724641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6e42e-9c93-41e1-a0d8-26fd19c80d02 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.768518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.769200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.769488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.770138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.771342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=778ab12e1e684f43aebaf63e47b6d6cf, user.id=e9d9ed1ce844447183749518f6265355) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:43.771671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.773650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6694e87-348d-4e09-9c9b-f2c90bcf2f1e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.782050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 98/193] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/role_assignments?user.id=e9d9ed1ce844447183749518f6265355&scope.project.id=778ab12e1e684f43aebaf63e47b6d6cf&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:43.783679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc92d8-87a5-4706-b277-8a3eb82f169a None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.783969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc92d8-87a5-4706-b277-8a3eb82f169a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.784414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc92d8-87a5-4706-b277-8a3eb82f169a None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.786341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 99/194] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:43.791686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc92d8-87a5-4706-b277-8a3eb82f169a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.845068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 96/195] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:43.846642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.851348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.851723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.872049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:43.874341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:43.930801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc92d8-87a5-4706-b277-8a3eb82f169a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:43.972369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:43.973269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:43.973554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:43.973834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] PATH_INFO: `/v3/users/17183b5e96664c19ac4edd822cdf8561` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:43.975658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] RBAC: Authorizing `identity:get_user(user_id=17183b5e96664c19ac4edd822cdf8561)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:43.977413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:43.980280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 100/196] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/users/17183b5e96664c19ac4edd822cdf8561 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:43.982911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:43.995443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 97/197] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:43.999866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.005354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.047093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.124466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405119bc-cfc5-4ccd-a42e-7398b7fe03e2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.163208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.164247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.164630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.164987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20f19-3648-40a7-acfd-87fa1168325d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.165104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.166361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.169000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.171118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 101/198] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:44.172053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.212612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.213769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.214197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.215005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.288635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.309914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.311834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.312039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010dfe0-c569-49a4-9bcd-e78eae56b7b3 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.344795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.356311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.357350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.357825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.362373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.384375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:44.393675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:44.394803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:44.395805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:44.402450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 102/199] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:44.402450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.459971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 98/200] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:44.461379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.522515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad239d26-b803-4b90-9403-c746c196da07 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.557802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.558509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.558929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.559300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.560234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=17183b5e96664c19ac4edd822cdf8561) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:44.561005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.562986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.569285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 103/201] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/role_assignments?user.id=17183b5e96664c19ac4edd822cdf8561&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:44.570260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.584665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.588804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ae1cd-0ba5-434f-b3e4-003d992479c8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.619044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.656689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.658843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.659230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.659622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.660496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:44.660902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.662567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.690399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 99/202] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:44.699829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.720646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd291a-7e28-4d53-85fc-1322f0afc06a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.766647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.768244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.768526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.768807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.770218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.772127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.774549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 104/203] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:44.781428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.881432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:44.886167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d003c-b5ad-440e-85fc-fc9d9da0dab9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.911605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236006bf-448d-4fb9-a050-4cd270b09a2c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:44.923106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.924276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.924556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.924831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] PATH_INFO: `/v3/users/6daf0a65684d416081460eac61880c28` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.926762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] RBAC: Authorizing `identity:get_user(user_id=6daf0a65684d416081460eac61880c28)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.928508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.930977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 100/204] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/users/6daf0a65684d416081460eac61880c28 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:44.948925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:44.957338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:44.961733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:44.962032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:44.962313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:44.963197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] RBAC: Adding query filter params (role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=17183b5e96664c19ac4edd822cdf8561) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:44.963512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:44.965038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:44.972022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 105/205] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/role_assignments?user.id=17183b5e96664c19ac4edd822cdf8561&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:45.024397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:45.074336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300e696-54a2-4b3b-a614-6747b47d0f22 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:45.118482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:45.119195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:45.119479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:45.119757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:45.212438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:45.213861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:45.274845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:45.336618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 101/206] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:45.342442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:45.448739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b71421-01b3-4c8a-bf6a-e0510255f3c1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:45.485016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b18ce7-624c-4adc-a030-32e09aa8d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:45.485819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b18ce7-624c-4adc-a030-32e09aa8d568 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:45.486101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b18ce7-624c-4adc-a030-32e09aa8d568 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:45.486432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b18ce7-624c-4adc-a030-32e09aa8d568 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:45.487714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b18ce7-624c-4adc-a030-32e09aa8d568 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:45.489231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b18ce7-624c-4adc-a030-32e09aa8d568 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:45.494813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 106/207] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:45 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:45.497278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:45.620554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a21ac-639c-43b5-b1b6-c481875da22c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:45.664504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:45.665172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:45.665452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:45.665726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:45.666569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] RBAC: Adding query filter params (role.id=a23a16fa358b491c876189b3d05e60c7, scope.project.id=536792d0280e402db36f3fcc605a2dbd, user.id=6daf0a65684d416081460eac61880c28) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:45.666908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:45.668378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac094-5eb7-4ce4-b9d6-1e22b5622ab6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:45.678827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 102/208] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:45 2022] GET /identity/v3/role_assignments?user.id=6daf0a65684d416081460eac61880c28&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=a23a16fa358b491c876189b3d05e60c7 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:45.867002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fe7c9-1b99-4bbc-a3bd-b5134033c66c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:45.867360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fe7c9-1b99-4bbc-a3bd-b5134033c66c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:45.867530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fe7c9-1b99-4bbc-a3bd-b5134033c66c None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:45.869192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 107/209] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:45.874990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca049c30-5b5c-4e2e-b68f-09693bcbe3d6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:45.875303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca049c30-5b5c-4e2e-b68f-09693bcbe3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:45.875551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca049c30-5b5c-4e2e-b68f-09693bcbe3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:45.894098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ca049c30-5b5c-4e2e-b68f-09693bcbe3d6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:45.895395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ca049c30-5b5c-4e2e-b68f-09693bcbe3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:45.911234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:45.971221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 103/210] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:45.995851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fe7c9-1b99-4bbc-a3bd-b5134033c66c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:46.006498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:46.078684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365b81a-1761-4123-b04b-bd0784d0aef2 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.079030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365b81a-1761-4123-b04b-bd0784d0aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.079268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365b81a-1761-4123-b04b-bd0784d0aef2 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.080676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 104/211] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:46.086002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.086290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.086569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.105967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:46.108073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:46.134900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fe7c9-1b99-4bbc-a3bd-b5134033c66c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:46.180682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:46.181507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.181788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.182060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.194385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:46.196729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 105/212] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:46.202222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:46.203152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:46.203923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 108/213] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:46.210461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:46.223720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:46.307821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:46.326389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:46.356298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4593fe-ba7c-4704-8294-a611779bb081 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:46.360310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14f8a9-e5a5-4cb4-b1e2-69699f474642 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:46.397768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:46.398763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.399225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.399674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.400630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:46.401114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:46.401440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:46.402332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.402685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.403058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.403592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:46.404851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:46.406362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:46.408884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 109/214] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:46.413916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:46.414226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 106/215] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:46.419773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:46.530204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865245f3-7456-4458-8e46-4fa5e7248d22 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:46.539867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3a1b9-07dc-4ee2-9a37-dbe2db3db73e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:46.566324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:46.567592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.568080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.568528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] PATH_INFO: `/v3/users/afae78d61ac24554afcac8d20bd954ec` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.579847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:46.583990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] RBAC: Authorizing `identity:get_user(user_id=afae78d61ac24554afcac8d20bd954ec)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:46.584140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:46.584963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.585315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.585691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.586555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:46.587483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:46.588244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:46.589601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 107/216] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:46 2022] GET /identity/v3/users/afae78d61ac24554afcac8d20bd954ec => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:46.596270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:46.596593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:46.616396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 110/217] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:46 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:46.720334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e86cb-e996-491a-b743-34cf0fb7f5f8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:46.761040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842a92c-3a04-4588-90bc-ad415b4994bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:46.761708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a92c-3a04-4588-90bc-ad415b4994bd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:46.761989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a92c-3a04-4588-90bc-ad415b4994bd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:46.762268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a92c-3a04-4588-90bc-ad415b4994bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:46.832879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:46.837160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842a92c-3a04-4588-90bc-ad415b4994bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:46.838930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842a92c-3a04-4588-90bc-ad415b4994bd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:46.944850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 108/218] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:02:46.950521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.067969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0c31-4cd4-4cec-b64f-55e0c8ef4626 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.068157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0c31-4cd4-4cec-b64f-55e0c8ef4626 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.068433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0c31-4cd4-4cec-b64f-55e0c8ef4626 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.071210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 109/219] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:47.075196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d008b-be88-4e56-ad41-a69071ac969a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.075486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d008b-be88-4e56-ad41-a69071ac969a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.075764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d008b-be88-4e56-ad41-a69071ac969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.077599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2b35f-1da0-4a1c-9098-17f6e880cb37 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:47.113565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:47.114455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.114760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.115047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.116292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:47.117869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:47.119645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 111/220] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:46 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:47.119766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-502d008b-be88-4e56-ad41-a69071ac969a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:47.121162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-502d008b-be88-4e56-ad41-a69071ac969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:47.125531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.187521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:47.234436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e12c2-a1a3-4055-951b-e1479c9acf50 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:47.285139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:47.285821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.286106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.286380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.287268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=afae78d61ac24554afcac8d20bd954ec) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:47.287587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:47.289053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e8bf5-5f78-4d32-a7e7-dbedea8400f5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:47.293976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 110/221] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:47.295079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 112/222] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:47 2022] GET /identity/v3/role_assignments?user.id=afae78d61ac24554afcac8d20bd954ec&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:47.296129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b370-1b8e-4dbf-9561-8bb2f70487cc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.296621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b370-1b8e-4dbf-9561-8bb2f70487cc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.297117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b370-1b8e-4dbf-9561-8bb2f70487cc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.298725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 111/223] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:47.304784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.305456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.305937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.322020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b370-1b8e-4dbf-9561-8bb2f70487cc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.343504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:47.347051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:47.368224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:47.456649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b370-1b8e-4dbf-9561-8bb2f70487cc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:47.500169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:47.501118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.501528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.501820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.510427 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 113/224] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:47.520760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:47.528934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:47.530098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:47.531337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 112/225] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:02:47.536742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.538157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.656783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:47.671713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d8c0a-d655-4539-8a56-22ae61b10260 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:47.695128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:47.711313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:47.712453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.712453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.712787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.724480 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:47.733491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:47.734405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:47.735229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 113/226] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:02:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:47.747910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.777707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e66b9-4d1f-4523-a611-962cb21f3b49 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:47.840099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:47.840785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:47.841068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:47.841345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:47.842144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:47.842450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:47.843880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:47.862736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 114/227] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:47.868047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:47.969922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302c5d3-ea1c-419d-893b-bf74cfd02e0b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.024613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8278c-cf8b-4a7e-9fbb-a930e6772e72 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.035358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.036031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.036308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.036574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.037336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:48.037692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.039131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.052638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:48.058327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 114/228] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:02:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:48.064132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:48.096247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.097199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.097524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.097797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] PATH_INFO: `/v3/users/17183b5e96664c19ac4edd822cdf8561` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.099875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] RBAC: Authorizing `identity:get_user(user_id=17183b5e96664c19ac4edd822cdf8561)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.104982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.109399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 115/229] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:47 2022] GET /identity/v3/users/17183b5e96664c19ac4edd822cdf8561 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:48.109742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:48.117406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:48.203142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903035e-69e6-4e90-942f-66ec64a2cd83 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.240873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.241929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.242212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.242481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] PATH_INFO: `/v3/users/17183b5e96664c19ac4edd822cdf8561` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.243971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] RBAC: Authorizing `identity:get_user(user_id=17183b5e96664c19ac4edd822cdf8561)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.245453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.247472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 115/230] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:48 2022] GET /identity/v3/users/17183b5e96664c19ac4edd822cdf8561 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:48.255681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:48.367110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:48.371480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f939-98df-4b9d-a663-5419d1ce31d5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.406400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc084b-5a0a-4a49-82fd-fdcaa995d2bf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.409660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.410802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.411288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.411807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.413206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.414929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.416871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 116/231] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:48 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:48.423185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:48.499372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:48.500055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.501485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.501485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.501485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.507708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.509334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.511368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 116/232] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:48 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:48.519169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:48.540538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cb892-b5f7-4389-be0b-f7c829978b3d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.575605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff029e-8ea7-4bda-b6fc-e8ccc81722a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.576959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff029e-8ea7-4bda-b6fc-e8ccc81722a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.577461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff029e-8ea7-4bda-b6fc-e8ccc81722a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.577940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff029e-8ea7-4bda-b6fc-e8ccc81722a5 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/17183b5e96664c19ac4edd822cdf8561/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.580194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff029e-8ea7-4bda-b6fc-e8ccc81722a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce6397fd8d4748b48261a3f3af2fc42a, user_id=17183b5e96664c19ac4edd822cdf8561, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.581904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff029e-8ea7-4bda-b6fc-e8ccc81722a5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.601095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 117/233] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:48 2022] PUT /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/17183b5e96664c19ac4edd822cdf8561/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:48.699250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce194d-7528-4238-b8fd-01fe3dadfa71 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:48.739113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c4c97-5a97-42f4-bc61-12d9c0bf529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:48.740031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c4c97-5a97-42f4-bc61-12d9c0bf529d None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:48.740312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c4c97-5a97-42f4-bc61-12d9c0bf529d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:48.740587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c4c97-5a97-42f4-bc61-12d9c0bf529d None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/17183b5e96664c19ac4edd822cdf8561/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:48.742932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c4c97-5a97-42f4-bc61-12d9c0bf529d None admin] RBAC: Authorizing `identity:create_grant(project_id=ce6397fd8d4748b48261a3f3af2fc42a, user_id=17183b5e96664c19ac4edd822cdf8561, role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:48.744435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c4c97-5a97-42f4-bc61-12d9c0bf529d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:48.771177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 117/234] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:48 2022] PUT /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/17183b5e96664c19ac4edd822cdf8561/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:02:49.211829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c110af-8e5b-4a14-8f76-f6dbd900ef34 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.212216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c110af-8e5b-4a14-8f76-f6dbd900ef34 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.212395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c110af-8e5b-4a14-8f76-f6dbd900ef34 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.214051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 118/235] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:49.220435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.220722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.221000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.238543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:49.244715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:49.246268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:49.336897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 118/236] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:49.382962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c110af-8e5b-4a14-8f76-f6dbd900ef34 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:49.526743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c110af-8e5b-4a14-8f76-f6dbd900ef34 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:49.538139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:49.565027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3d5d-6473-48e9-b2a6-3bd4d2fe1c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:49.565854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3d5d-6473-48e9-b2a6-3bd4d2fe1c5d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.566137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3d5d-6473-48e9-b2a6-3bd4d2fe1c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.566412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3d5d-6473-48e9-b2a6-3bd4d2fe1c5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.577711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d3d5d-6473-48e9-b2a6-3bd4d2fe1c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-943d3d5d-6473-48e9-b2a6-3bd4d2fe1c5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:49.585511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:49.586386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:49.587221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:49.590811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 119/237] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:49.593253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:49.608755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311aee7-1181-49d5-a43f-2b57ddef50a3 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.609040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311aee7-1181-49d5-a43f-2b57ddef50a3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.609312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311aee7-1181-49d5-a43f-2b57ddef50a3 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.610719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 120/238] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:49.615798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553cca9-e800-4e7d-897d-f2f915912ae4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.616087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553cca9-e800-4e7d-897d-f2f915912ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.616362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553cca9-e800-4e7d-897d-f2f915912ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.634120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1553cca9-e800-4e7d-897d-f2f915912ae4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:49.635366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1553cca9-e800-4e7d-897d-f2f915912ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:49.712928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34627a-9015-45e9-91fb-a612514cb11e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:49.717755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 121/239] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:49.719204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84487c-8e23-4613-85ac-aa468051987f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.719556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84487c-8e23-4613-85ac-aa468051987f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.719900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84487c-8e23-4613-85ac-aa468051987f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.721392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 122/240] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:49.726654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.728775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.729236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.747141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:49.748629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:49.757553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:49.758558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.759107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.759451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.760296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:49.760670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:49.762148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:49.764835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:49.772186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 119/241] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:49.773193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:49.833946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 123/242] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:49.835283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:49.870739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:49.908340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215442ba-e802-4b20-98a5-cefbc5b6f816 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:49.954854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5980d3e-5e1f-47d3-bd8b-d2cf240492db None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:49.955060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:49.955844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.956122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.956397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:49.967928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:49.975936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:49.976818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:49.977583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:49.982922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 120/243] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:49.983973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:49.992248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:49.993195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:49.993581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:49.993865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] PATH_INFO: `/v3/users/e7c170c590c349259888516094d422a2` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.010992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] RBAC: Authorizing `identity:get_user(user_id=e7c170c590c349259888516094d422a2)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.012617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.014745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 124/244] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:49 2022] GET /identity/v3/users/e7c170c590c349259888516094d422a2 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.015576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.050175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.093954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa66781-1296-4425-b3db-6cc5a0e22d0f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.112512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.127013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39cdec-1d16-431c-89cb-b86e72f5f41b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.130571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.131493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.131874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.132216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.133464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.135035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.137183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 121/245] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.138123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.169053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.169738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.170083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.170364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.171158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:50.171469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.173061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.183722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 125/246] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.184537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.256405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f54eea-305e-489c-88f2-ff5944dc1d9d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.284341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.294534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.296256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.296543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.296907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.299930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397876a-32f3-4d48-9d12-7a341a57fb5f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.336947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.337665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.337952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.338228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.338988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.340650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.370985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 126/247] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:50 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:50.371830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.377172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.379023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.395009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.553606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d358-38e1-40c2-8bff-dc59e3ae40f2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.585131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 122/248] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:50.591912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.596944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.597856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.598139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.598414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] PATH_INFO: `/v3/users/afae78d61ac24554afcac8d20bd954ec` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.599676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] RBAC: Authorizing `identity:get_user(user_id=afae78d61ac24554afcac8d20bd954ec)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.601143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.602815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 127/249] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:50 2022] GET /identity/v3/users/afae78d61ac24554afcac8d20bd954ec => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.610032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.627643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.711290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.718357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d673222-a18e-4b95-a953-3c303e15f8c2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.740420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe45dc-c570-4a48-9247-7ff89fe7fc8a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.759924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.761108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.761584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.762046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.764674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.767129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.775801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 123/250] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:50 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.777085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.786958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.788069 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.788493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.788906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.790254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.791943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.794218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 128/251] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:50 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.799628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:50.904672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:50.912877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35bf34-3e0f-4e1f-9bcc-ba8a3133a977 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.951136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac050a11-e70a-4f3d-832e-e15cd691aff5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:50.958093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:50.958831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.959605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.959889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.960738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] RBAC: Adding query filter params (role.id=a23a16fa358b491c876189b3d05e60c7, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=e7c170c590c349259888516094d422a2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:50.961048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:50.962626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b19e2c-b577-45ef-afec-b82e0b77d0ea None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:50.968840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 124/252] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:50 2022] GET /identity/v3/role_assignments?user.id=e7c170c590c349259888516094d422a2&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=a23a16fa358b491c876189b3d05e60c7 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:50.970122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed175552-9f39-4b03-a896-b959278a2a0f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.970406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed175552-9f39-4b03-a896-b959278a2a0f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.970676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed175552-9f39-4b03-a896-b959278a2a0f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.972072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 125/253] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:50.977417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:50.977819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:50.978101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:50.998439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:50.999831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:51.015583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:51.016632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.016938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.017206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/afae78d61ac24554afcac8d20bd954ec/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.025513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce6397fd8d4748b48261a3f3af2fc42a, user_id=afae78d61ac24554afcac8d20bd954ec, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:51.027083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:51.048712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 129/254] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:50 2022] PUT /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/afae78d61ac24554afcac8d20bd954ec/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:02:51.096606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 126/255] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:51.137302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:51.140631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:51.270942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899ee5a-219b-4cd5-9165-a4fd97d82796 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:51.305980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c10e2-ac89-4a5a-a7d3-6cea84000ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:51.306827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c10e2-ac89-4a5a-a7d3-6cea84000ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.307113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c10e2-ac89-4a5a-a7d3-6cea84000ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.307390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c10e2-ac89-4a5a-a7d3-6cea84000ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.318534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c10e2-ac89-4a5a-a7d3-6cea84000ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-470c10e2-ac89-4a5a-a7d3-6cea84000ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:51.326659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:51.327568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:51.328351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 130/256] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:51.333890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:51.403979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20912849-5ded-41ce-b37a-19688077bfc3 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.404459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20912849-5ded-41ce-b37a-19688077bfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.404694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20912849-5ded-41ce-b37a-19688077bfc3 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.406323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 131/257] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:02:51.413662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.413944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.414388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.442440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:51.443700 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:51.455869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:51.487128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0f22-a2e7-4c44-9c09-bb6e8dbf51eb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:51.557018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:51.557611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.557895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.558173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.558973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:51.559282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:51.561303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:51.577567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:51.579879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 127/258] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:51.587001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:51.610841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 132/259] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:51.672257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:51.757476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32610b3-9d65-4ef3-8286-56262faedecd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:51.808746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:51.809556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.809836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.810111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] PATH_INFO: `/v3/users/17183b5e96664c19ac4edd822cdf8561` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.812301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] RBAC: Authorizing `identity:get_user(user_id=17183b5e96664c19ac4edd822cdf8561)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:51.813853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:51.815584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 128/260] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:51 2022] GET /identity/v3/users/17183b5e96664c19ac4edd822cdf8561 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:51.823008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:51.855293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbec39d-176e-4c23-91d1-c091357ecffe None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:51.860743 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:51.926206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:51.927064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:51.927348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:51.927617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:51.951032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:51.959549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:51.961875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:51.963816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 133/261] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:02:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:02:51.969370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:52.032976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cb0cb-6139-4c61-ae26-5d837dc0ac35 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:52.065292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:52.151975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:52.152650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:52.152934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:52.153208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:52.156855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4d20-0ddd-4c18-8971-e0ef437dacb2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:52.220624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:52.221313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:52.221646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:52.221929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:52.222696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:52.223170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:52.224811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:52.240376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 134/262] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:02:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:52.246208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:52.369330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:52.372455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:52.404562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:52.417821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07febf-8985-4675-a0b9-7fd778021c4b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:52.446160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:52.493980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:52.495292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:52.495743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:52.496153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] PATH_INFO: `/v3/users/17183b5e96664c19ac4edd822cdf8561` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:52.497967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] RBAC: Authorizing `identity:get_user(user_id=17183b5e96664c19ac4edd822cdf8561)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:52.500527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:52.502815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 135/263] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:52 2022] GET /identity/v3/users/17183b5e96664c19ac4edd822cdf8561 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:52.518680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 129/264] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:02:52.519920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:52.526071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:52.676663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f9cf0-318b-4bec-a923-d5b3c60f39ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:52.678555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec685d9f-9f48-412d-9d53-ec921443ebb8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:52.715883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:52.723934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:52.724784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:52.726362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:52.726362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:52.784294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:52.785767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:52.786184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:52.786622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:52.787969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:52.789571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:52.799850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 130/265] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:52 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:52.802848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:52.815671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:52.834727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:52.843410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:52.964547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 136/266] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:52.971965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:53.080705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd129e7f-8236-4545-89e7-e405b166f7c3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:53.112827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:53.133831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd34ef-826e-4b33-b920-51b3188d8375 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:53.143001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:53.143737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.144065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.144343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.145207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=17183b5e96664c19ac4edd822cdf8561) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:53.145526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:53.147036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:53.155282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 131/267] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:52 2022] GET /identity/v3/role_assignments?user.id=17183b5e96664c19ac4edd822cdf8561&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:53.186604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56178c3b-f9ee-4262-9a9c-4c603fa9f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:53.187535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178c3b-f9ee-4262-9a9c-4c603fa9f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.187817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178c3b-f9ee-4262-9a9c-4c603fa9f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.188091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178c3b-f9ee-4262-9a9c-4c603fa9f9c2 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.189300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178c3b-f9ee-4262-9a9c-4c603fa9f9c2 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:53.190805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178c3b-f9ee-4262-9a9c-4c603fa9f9c2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:53.192553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 137/268] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:52 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:53.199057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:53.272284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:53.350516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6d9f9-e9c1-4f82-8e92-2beade94659b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:53.397204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:53.397890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.398174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.398452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.399345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] RBAC: Adding query filter params (role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=17183b5e96664c19ac4edd822cdf8561) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:53.399658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:53.401142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06bebe1-ed61-4c99-9727-b4457ec94fae None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:53.408354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 132/269] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:53 2022] GET /identity/v3/role_assignments?user.id=17183b5e96664c19ac4edd822cdf8561&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:53.821716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fa55e-4a50-4c95-9dcc-98fdf8c42d08 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.822258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fa55e-4a50-4c95-9dcc-98fdf8c42d08 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.822258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fa55e-4a50-4c95-9dcc-98fdf8c42d08 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.823816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 138/270] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:53.830652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caa9ff-dc27-4963-afdb-c19d4acc8fc2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.831150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caa9ff-dc27-4963-afdb-c19d4acc8fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.831246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caa9ff-dc27-4963-afdb-c19d4acc8fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.852261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-78caa9ff-dc27-4963-afdb-c19d4acc8fc2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:53.853441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-78caa9ff-dc27-4963-afdb-c19d4acc8fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:53.859431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738d8f5-4375-40cb-af39-f39368699e31 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.859997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738d8f5-4375-40cb-af39-f39368699e31 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.860489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738d8f5-4375-40cb-af39-f39368699e31 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.862337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 139/271] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:53.867778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.868320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.868791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.890918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:53.892485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:53.943137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 133/272] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:53.943137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b4d35-2e08-466a-9e3b-23f10f230e81 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.943137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b4d35-2e08-466a-9e3b-23f10f230e81 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.943137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b4d35-2e08-466a-9e3b-23f10f230e81 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.943547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 134/273] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:53.949020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:53.949339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:53.949688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:53.967929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:53.969042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:53.970841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 140/274] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:53.972595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.014687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.044206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 135/275] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:54.045525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.064335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.088711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba83ec-210b-4a4e-877a-367314eecccd None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.123856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.124663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.124950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.125223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.135247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:54.143147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:54.144248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:54.145123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 141/276] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:54.149806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.163706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e56ff-42dd-4017-9ead-577833315f10 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.197315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.198110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.198386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.198659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.199846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.201349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.207512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 136/277] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:54.207512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.250551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.254283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f823a94-3f98-4ebc-b8ee-1b35604edfc0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.289039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4a5d9-147d-4f72-b982-9f58a09bbc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.289850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4a5d9-147d-4f72-b982-9f58a09bbc0f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.290135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4a5d9-147d-4f72-b982-9f58a09bbc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.290408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4a5d9-147d-4f72-b982-9f58a09bbc0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.298284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.301960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4a5d9-147d-4f72-b982-9f58a09bbc0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:54.306983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851283-d54d-4d7b-9bac-85834553f92c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-91f4a5d9-147d-4f72-b982-9f58a09bbc0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:54.309864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:54.310894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:54.311943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 142/278] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:54.313048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52893a7-8fde-4a46-ae53-9369fcbac1f3 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.313048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52893a7-8fde-4a46-ae53-9369fcbac1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.313048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52893a7-8fde-4a46-ae53-9369fcbac1f3 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.314211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 143/279] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:54.314908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52893a7-8fde-4a46-ae53-9369fcbac1f3 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.345587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.346258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.346541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.346865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.347631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:54.347943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.349356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.357808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 137/280] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:54.358603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.424870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52893a7-8fde-4a46-ae53-9369fcbac1f3 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.456588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16147995-e9bb-47cb-993f-304cd75d6b43 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.459462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.460322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d03f5-5f71-4bca-95a9-4bc06ba461f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.461070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d03f5-5f71-4bca-95a9-4bc06ba461f2 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.461355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d03f5-5f71-4bca-95a9-4bc06ba461f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.461797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d03f5-5f71-4bca-95a9-4bc06ba461f2 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.462527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d03f5-5f71-4bca-95a9-4bc06ba461f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.464230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d03f5-5f71-4bca-95a9-4bc06ba461f2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.485487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 144/281] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:02:54 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:02:54.486770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.487060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.487336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.505166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:54.506328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:54.509896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.510610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.510953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.511219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.512016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:54.512317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.513949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.524439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 138/282] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:54.529979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.565777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.586695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 145/283] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:54.591687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.644180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226a599-9d92-4b8a-963e-f3284e603ad1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.685464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.686487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.687373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.687528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] PATH_INFO: `/v3/users/d777c2ceaa0243459d60f55d7254f1e5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.708311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=d777c2ceaa0243459d60f55d7254f1e5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.709808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.711771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 139/284] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/users/d777c2ceaa0243459d60f55d7254f1e5 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:54.716790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.725912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff747c-6bcb-4bac-baa4-de0be6ce0ef8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.744009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.771633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.772452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.772733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.773681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] PATH_INFO: `/v3/users/afae78d61ac24554afcac8d20bd954ec` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.774256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] RBAC: Authorizing `identity:get_user(user_id=afae78d61ac24554afcac8d20bd954ec)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.775793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.782306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 146/285] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/users/afae78d61ac24554afcac8d20bd954ec => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:54.782306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.846488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:54.869033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b64f3c-1eed-4eb2-8fd1-c8c622c97bb7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.894051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde1857-4e5d-4ce3-a5ac-0ff2b2bf88a9 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:54.913417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.914291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.914577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.914880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.929641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:54.930198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:54.930909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:54.931191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:54.931467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:54.932305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, user.id=d777c2ceaa0243459d60f55d7254f1e5, scope.system=all) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:54.932617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:54.934079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:54.938403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:54.939340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:54.940606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 140/286] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:54.942351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 147/287] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/role_assignments?user.id=d777c2ceaa0243459d60f55d7254f1e5&scope.system=all&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:54.942500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:54.946242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:55.045112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:55.072081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7900c-5a77-4b4a-8612-962fb1c3f487 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.088546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1f346-5955-4578-8f8b-5b095bdba763 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.109239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:55.123741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.124409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.124688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.124952 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.146077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.147610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.148102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.148578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.149554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:55.150088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.151776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.168133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 148/288] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:55.174307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:55.227324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.229157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.297767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d9fc8-b846-480e-acfe-2f2be931f91c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.315155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:55.334400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 141/289] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:55.342963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.342963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.342963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.342963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] PATH_INFO: `/v3/users/e7c170c590c349259888516094d422a2` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.343315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] RBAC: Authorizing `identity:get_user(user_id=e7c170c590c349259888516094d422a2)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.343315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.344242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 149/290] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:55 2022] GET /identity/v3/users/e7c170c590c349259888516094d422a2 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:55.344818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:55.351627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:55.443687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:55.449131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398aafd-e2e7-41f7-bf0b-79d16bc499fa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.502690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.505312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.505822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.507095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.509267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.511131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.518334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 142/291] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:55 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:55.527673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:55.534302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0ecd-ab07-498c-8d62-a45c2c493630 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.570297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.571271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.571564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.571844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.576178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.576178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.576872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 150/292] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:55 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:55.582196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:55.637233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:55.728340 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2303a-2a32-4cdd-afaf-bd697eeac809 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.747378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:55.749698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986c249-df33-40dc-b6bb-942bb7b4505c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:55.778423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9a5f2-ea3d-4233-be84-331c5bbeeee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.779430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9a5f2-ea3d-4233-be84-331c5bbeeee4 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.779718 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9a5f2-ea3d-4233-be84-331c5bbeeee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.779996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9a5f2-ea3d-4233-be84-331c5bbeeee4 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/e7c170c590c349259888516094d422a2/roles/a23a16fa358b491c876189b3d05e60c7` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.785460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9a5f2-ea3d-4233-be84-331c5bbeeee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce6397fd8d4748b48261a3f3af2fc42a, user_id=e7c170c590c349259888516094d422a2, role_id=a23a16fa358b491c876189b3d05e60c7)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.786955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9a5f2-ea3d-4233-be84-331c5bbeeee4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.808871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 151/293] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:02:55 2022] PUT /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/users/e7c170c590c349259888516094d422a2/roles/a23a16fa358b491c876189b3d05e60c7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:02:55.812599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:55.813341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:55.813626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:55.813903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:55.814796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=afae78d61ac24554afcac8d20bd954ec) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:55.815186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:55.816883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf557244-5e08-4c9a-b510-3571c6c867b1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:55.824115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 143/294] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:55 2022] GET /identity/v3/role_assignments?user.id=afae78d61ac24554afcac8d20bd954ec&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:56.855010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e369b-6c3f-4f3b-8b5c-88391d5e220e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:56.855536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e369b-6c3f-4f3b-8b5c-88391d5e220e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:56.855536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e369b-6c3f-4f3b-8b5c-88391d5e220e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:56.857196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 152/295] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:56.863144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0b902-fc20-4c10-9f0f-e3b9b6e55a37 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:56.863599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0b902-fc20-4c10-9f0f-e3b9b6e55a37 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:56.863672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0b902-fc20-4c10-9f0f-e3b9b6e55a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:56.882348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-61b0b902-fc20-4c10-9f0f-e3b9b6e55a37 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:56.883599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-61b0b902-fc20-4c10-9f0f-e3b9b6e55a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:56.957258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 144/296] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:56.972401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054efa4-47bb-4fa8-a070-5ee5645f6702 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:56.972960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054efa4-47bb-4fa8-a070-5ee5645f6702 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:56.973436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054efa4-47bb-4fa8-a070-5ee5645f6702 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:56.975307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 153/297] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:56.980542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054efa4-47bb-4fa8-a070-5ee5645f6702 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:56.981185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:56.981484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:56.981760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:56.999946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:57.001142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:57.047788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:57.082926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 145/298] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:57.098443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.112532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054efa4-47bb-4fa8-a070-5ee5645f6702 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.162632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.164042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.164734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.165212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.167230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:57.168874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:57.171052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 154/299] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:57.177840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.212978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64436608-2152-4262-8fce-1c5bb273b79c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.239547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:57.261227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.263022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.263534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.264228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.277017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:57.288014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:57.288912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:57.289708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:02:57.297351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 146/300] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:57.297972 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.314644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422230f-a41f-4297-81c6-d82e6e77edd8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.342744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:57.357426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f9396-2140-4189-8ac5-5e3f017f242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.358605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f9396-2140-4189-8ac5-5e3f017f242c None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.358931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f9396-2140-4189-8ac5-5e3f017f242c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.359208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f9396-2140-4189-8ac5-5e3f017f242c None admin] PATH_INFO: `/v3/groups` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.360084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f9396-2140-4189-8ac5-5e3f017f242c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:57.362183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f9396-2140-4189-8ac5-5e3f017f242c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:57.383452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 155/301] 10.222.0.64 () {60 vars in 1134 bytes} [Fri Jun 24 10:02:57 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:02:57.404255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1428a-a947-4f56-9a10-7381db80142e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.404580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1428a-a947-4f56-9a10-7381db80142e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.405556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1428a-a947-4f56-9a10-7381db80142e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.407386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 156/302] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:57.413072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.413399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.413684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.432509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:57.433830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:57.434215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200d18-aa0c-4884-aac9-54157416cbbb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.482584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.483791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.484865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.485370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.486780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:57.487550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:57.489357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:57.501343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 147/303] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:57.507445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.527087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 157/304] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:57.544186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.585270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:57.634308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:57.638893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074f8b-4fef-4c53-97e9-36dabe22ceb4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.674747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.676037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.676530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.676905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] PATH_INFO: `/v3/users/7eed777e01bd48fc8c14e5a86af80e39` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.679195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7940f-6732-4521-8cb6-36dd98e0b77f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.695734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] RBAC: Authorizing `identity:get_user(user_id=7eed777e01bd48fc8c14e5a86af80e39)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:57.697397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:57.699619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 148/305] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/users/7eed777e01bd48fc8c14e5a86af80e39 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:57.705095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.735961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.737456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.738134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.738547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.753142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:57.764100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:57.765547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:02:57.766532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:57.769115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 158/306] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:57.775047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.835388 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a911cb9-9c54-4d26-8927-5dd069d1488b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.882124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.883732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.884433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.884879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.886228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] RBAC: Adding query filter params (role.id=a23a16fa358b491c876189b3d05e60c7, user.id=7eed777e01bd48fc8c14e5a86af80e39, scope.system=all) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:57.886727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:57.889598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:57.895089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:57.899339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 149/307] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/role_assignments?user.id=7eed777e01bd48fc8c14e5a86af80e39&scope.system=all&role.id=a23a16fa358b491c876189b3d05e60c7 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:57.907014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2967-d2cb-43b5-a3f3-e381f2455d09 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:57.946484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:57.947257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.947539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.947817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.948597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:57.948909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:57.950370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169b533-c24b-49a2-be9d-ea41676bf649 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:57.962935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 159/308] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:02:57.971951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:57.988185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38874878-0b1a-4ec6-942f-151b9a74ba1e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.988811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38874878-0b1a-4ec6-942f-151b9a74ba1e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.989378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38874878-0b1a-4ec6-942f-151b9a74ba1e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:57.991505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 160/309] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:57.996994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:57.997609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:57.998229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:58.002286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:58.022428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:58.024026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:58.085748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d120d3-3f84-4c52-9ec1-f7e25dd8009d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:58.102666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 161/310] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:02:58.124161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:58.125179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:58.125436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:58.125786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] PATH_INFO: `/v3/users/d777c2ceaa0243459d60f55d7254f1e5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:58.127136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] RBAC: Authorizing `identity:get_user(user_id=d777c2ceaa0243459d60f55d7254f1e5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:58.127261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:58.128622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:58.136625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 150/311] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:57 2022] GET /identity/v3/users/d777c2ceaa0243459d60f55d7254f1e5 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:58.137975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:58.266719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:58.288734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e80015-d431-40e8-b7da-da9d1a1afa6c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:58.364879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:58.365702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:58.370736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:58.370736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:58.382624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:58.386586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:02:58.394793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:02:58.395686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:02:58.397098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 162/312] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:02:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:02:58.402626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:58.415676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ab1b3-9454-49a4-9048-ef842a0ecfe4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:58.475811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:58.476926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:58.478235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:58.478923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] PATH_INFO: `/v3/system/users/d777c2ceaa0243459d60f55d7254f1e5/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:58.481843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d777c2ceaa0243459d60f55d7254f1e5, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:58.484467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:58.505647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 151/313] 10.222.0.64 () {58 vars in 1334 bytes} [Fri Jun 24 10:02:58 2022] PUT /identity/v3/system/users/d777c2ceaa0243459d60f55d7254f1e5/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:02:58.636917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb17420-08d8-4bd9-b4aa-12ce07bb6d5b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:58.675920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:58.676915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:58.677355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:58.677773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:58.678760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:58.679222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:58.680781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:58.691305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 163/314] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:02:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:58.702737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:58.799261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6f13-e5c1-4a28-b648-2a824e2ee511 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:58.833663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:58.834554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:58.834860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:58.835137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] PATH_INFO: `/v3/users/e7c170c590c349259888516094d422a2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:58.836332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] RBAC: Authorizing `identity:get_user(user_id=e7c170c590c349259888516094d422a2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:58.837814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:58.839801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 152/315] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:02:58 2022] GET /identity/v3/users/e7c170c590c349259888516094d422a2 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:58.848044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:58.860780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:58.953816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f1da7-09f1-4a0d-a747-d44d4a1df246 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:58.989129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:58.990068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:58.990497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:58.990954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:59.062439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:59.064178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:59.074653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:59.172457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 164/316] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:02:59.182726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:59.200794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:59.287967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec4892-e419-4475-bcb2-94c81b4536f6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:59.322201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42960cfb-c14d-4064-a554-377c1e1e663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:59.323102 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42960cfb-c14d-4064-a554-377c1e1e663a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:59.323382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42960cfb-c14d-4064-a554-377c1e1e663a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:59.323656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42960cfb-c14d-4064-a554-377c1e1e663a None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:59.324838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42960cfb-c14d-4064-a554-377c1e1e663a None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:59.326344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42960cfb-c14d-4064-a554-377c1e1e663a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:59.328413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 153/317] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:02:59 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:59.334138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:59.437163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f7398-4a62-47fb-baba-d2a8bfcd5e71 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:59.447464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:59.473166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:59.474221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:59.474735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:59.475234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:59.476275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] RBAC: Adding query filter params (role.id=a23a16fa358b491c876189b3d05e60c7, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a, user.id=e7c170c590c349259888516094d422a2) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:02:59.476788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:59.478541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a054ad1-eb2f-465b-9854-70d0c3c3821b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:59.486093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 165/318] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:02:59 2022] GET /identity/v3/role_assignments?user.id=e7c170c590c349259888516094d422a2&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=a23a16fa358b491c876189b3d05e60c7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:59.588607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2034bc-113d-44b4-8bff-64c32914c83f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:59.588780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2034bc-113d-44b4-8bff-64c32914c83f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:59.589064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2034bc-113d-44b4-8bff-64c32914c83f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:59.590786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 154/319] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:02:59.596708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:59.597020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:59.597272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:59.616856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:02:59.618095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:02:59.693105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 166/320] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:02:59.714941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2034bc-113d-44b4-8bff-64c32914c83f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:59.810628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:02:59.824619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2034bc-113d-44b4-8bff-64c32914c83f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:59.859989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e67253-8105-421f-9ec2-f6c97237be5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:02:59.860695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67253-8105-421f-9ec2-f6c97237be5b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:02:59.860967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67253-8105-421f-9ec2-f6c97237be5b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:02:59.861240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67253-8105-421f-9ec2-f6c97237be5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:02:59.862408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e67253-8105-421f-9ec2-f6c97237be5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:02:59.863908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e67253-8105-421f-9ec2-f6c97237be5b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:02:59.865703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 155/321] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:02:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:02:59.871234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:02:59.974207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2df4cc-cc33-40d6-b8df-483359ccc61c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:02:59.975743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:00.010055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bd23f-2d36-4395-acaf-33e436f3dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:00.010754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd23f-2d36-4395-acaf-33e436f3dddb None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.011032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd23f-2d36-4395-acaf-33e436f3dddb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.011307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd23f-2d36-4395-acaf-33e436f3dddb None admin] PATH_INFO: `/v3/groups` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.012000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bd23f-2d36-4395-acaf-33e436f3dddb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:00.013464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bd23f-2d36-4395-acaf-33e436f3dddb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:00.038731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 167/322] 10.222.0.64 () {60 vars in 1134 bytes} [Fri Jun 24 10:02:59 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:00.068641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df1d00-9a78-4f08-a1dc-7806f6aa0b1d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.068977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df1d00-9a78-4f08-a1dc-7806f6aa0b1d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.069301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df1d00-9a78-4f08-a1dc-7806f6aa0b1d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.070797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 156/323] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:00.076839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.077183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.077360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.095728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:00.096959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:00.179139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 168/324] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:00.197111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df1d00-9a78-4f08-a1dc-7806f6aa0b1d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:00.311008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df1d00-9a78-4f08-a1dc-7806f6aa0b1d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:00.349267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:00.350444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.350902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.351378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.363840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:00.371931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:00.373190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:03:00.374173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:00.377010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 157/325] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:00.381694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:00.485575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:00.497144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96291a2e-442e-4d09-807c-c6baf01ed7fa None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:00.538850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:00.539540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.539821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.540100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.540863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:00.541172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:00.542631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5810-d8d2-4072-89ea-9fe5994dd21f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:00.555824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 169/326] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:00.558545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:00.579269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:00.674160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01aa7b-ab70-4df4-860e-ff685542e744 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:00.676396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185e2cc-453b-40e3-8e8a-871d70eb025d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.676690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185e2cc-453b-40e3-8e8a-871d70eb025d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.676969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185e2cc-453b-40e3-8e8a-871d70eb025d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.678529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 170/327] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:00.683883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.684145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.684439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.716528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:00.718263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:00.721142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:00.722067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.722349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.722603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] PATH_INFO: `/v3/users/7eed777e01bd48fc8c14e5a86af80e39` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.723910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] RBAC: Authorizing `identity:get_user(user_id=7eed777e01bd48fc8c14e5a86af80e39)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:00.725452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:00.733221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 158/328] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:03:00 2022] GET /identity/v3/users/7eed777e01bd48fc8c14e5a86af80e39 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:00.734448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:00.796796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 171/329] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:00.823278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:00.838565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:00.861357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6d40f-5618-41ff-baa5-20ed31b61094 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:00.910545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:00.911713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:00.912144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:00.912418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] PATH_INFO: `/v3/system/users/7eed777e01bd48fc8c14e5a86af80e39/roles/a23a16fa358b491c876189b3d05e60c7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:00.914821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7eed777e01bd48fc8c14e5a86af80e39, role_id=a23a16fa358b491c876189b3d05e60c7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:00.916221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:00.929787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 159/330] 10.222.0.64 () {58 vars in 1334 bytes} [Fri Jun 24 10:03:00 2022] PUT /identity/v3/system/users/7eed777e01bd48fc8c14e5a86af80e39/roles/a23a16fa358b491c876189b3d05e60c7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:03:00.951667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:00.975298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f02e5-388b-4843-86ac-ddf62875443f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:01.029769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:01.031060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:01.031492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:01.031908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:01.047125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:01.055507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:01.056651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:01.057652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:01.061686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 172/331] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:01.067763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:01.203943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a2428-6d0a-4833-b2e2-bffd0ddcfd55 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:01.247146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:01.248169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:01.248601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:01.249016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:01.249970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:01.250434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:01.252110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:01.267329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 160/332] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:01.272569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:01.390058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65506a8-1607-4ab7-8733-ec9072aa64e4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:01.436337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65f912-a5a4-4578-a334-722f02e3b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:01.437524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f912-a5a4-4578-a334-722f02e3b60f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:01.437953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f912-a5a4-4578-a334-722f02e3b60f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:01.438372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f912-a5a4-4578-a334-722f02e3b60f None admin] PATH_INFO: `/v3/users/d777c2ceaa0243459d60f55d7254f1e5` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:01.439783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65f912-a5a4-4578-a334-722f02e3b60f None admin] RBAC: Authorizing `identity:get_user(user_id=d777c2ceaa0243459d60f55d7254f1e5)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:01.441414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65f912-a5a4-4578-a334-722f02e3b60f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:01.446800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 173/333] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:03:01 2022] GET /identity/v3/users/d777c2ceaa0243459d60f55d7254f1e5 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:01.449174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:01.463087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:01.568942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad133-4730-4136-a6fd-233b53d69839 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:01.610770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:01.611497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:01.611782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:01.612059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:01.612901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] RBAC: Adding query filter params (role.id=0f531bdb05494dcf9bf516de1fb46554, user.id=d777c2ceaa0243459d60f55d7254f1e5, scope.system=all) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:01.613214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:01.614788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57a110-ed5c-47ed-a870-c602c43e8032 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:01.634719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 161/334] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Jun 24 10:03:01 2022] GET /identity/v3/role_assignments?user.id=d777c2ceaa0243459d60f55d7254f1e5&scope.system=all&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:02.583426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995fe95-69d9-4ede-b6ca-84a097b1ebfe None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:02.583960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995fe95-69d9-4ede-b6ca-84a097b1ebfe None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:02.583960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995fe95-69d9-4ede-b6ca-84a097b1ebfe None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:02.585371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 174/335] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:02.590766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:02.591185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:02.591365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:02.611043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:02.612263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:02.686106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:02.692251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 162/336] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:02.736441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995fe95-69d9-4ede-b6ca-84a097b1ebfe None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:02.752798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:02.856672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995fe95-69d9-4ede-b6ca-84a097b1ebfe None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:02.902289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:02.903168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:02.903455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:02.903731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:02.920670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:02.930695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:02.931606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:02.932470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:02.933059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 175/337] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:02.939596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:03.050946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215ac70-646d-4565-85b0-c3356978a1b6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:03.085600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:03.086274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.086555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.086852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.087608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:03.087917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:03.089317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffff75-43ef-4c01-ae76-236fb056b715 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:03.098348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 163/338] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:03.103600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:03.191995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:03.241190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d69f3-ceff-4aeb-872d-7eac256c3090 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:03.250472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719902b-8c05-407a-97ba-d657250a70b2 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.250472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719902b-8c05-407a-97ba-d657250a70b2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.250472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719902b-8c05-407a-97ba-d657250a70b2 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.250882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 164/339] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:03.256804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.257090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.257370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.289132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:03.289802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.290084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.290360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.293287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:03.294691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:03.371331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:03.373566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:03.378292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:03.397856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 165/340] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:03.432010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:03.450447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:03.486531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 176/341] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:03.494282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:03.602071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b7aae-3f9b-4a2d-a338-2e381fe7e723 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:03.643670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:03.644914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.644914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.645218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.647141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:03.648996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:03.654047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 177/342] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:03 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:03.665473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9623-5a59-460b-a372-a93b834e5bf3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:03.669775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:03.751448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:03.752280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.752556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.752895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.753886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:03.776906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:03.785246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:03.786132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:03.787018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 166/343] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:03.798866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:03.862498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf30cfd-3e1a-43b7-92bb-65a2c527c847 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:03.900217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:03.936816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:03.938154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:03.938641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:03.939165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:03.957414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:03.960059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:03.962250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 178/344] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:03 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:03.971690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:04.009073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11e5d-27e6-4eba-a257-68c6ccb6c5e5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:04.073711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:04.074815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.075300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.075773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.076817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:04.077378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:04.078993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:04.091781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 167/345] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:04.098290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:04.163002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499d49c-0917-4554-ab4f-c61802f2bbc4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:04.167007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:04.232429 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:04.233063 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.233348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.233629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.234488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=536792d0280e402db36f3fcc605a2dbd) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:04.234830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:04.236298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03d9a-77d8-4c91-b652-66fbc175028d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:04.248688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 179/346] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:03 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:04.250030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1bbed-0930-451b-b570-4cb4a163667e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.250320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1bbed-0930-451b-b570-4cb4a163667e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.250597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1bbed-0930-451b-b570-4cb4a163667e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.252092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 180/347] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:04.257639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c40493-aa1c-4455-a733-735a9e550ca5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.257934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c40493-aa1c-4455-a733-735a9e550ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.258228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c40493-aa1c-4455-a733-735a9e550ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.273344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09048e15-c594-406b-aa66-cfc0448b4a03 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:04.302558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-51c40493-aa1c-4455-a733-735a9e550ca5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:04.303280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:04.304106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-51c40493-aa1c-4455-a733-735a9e550ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:04.328061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:04.328726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.329009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.329287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.446064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 181/348] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:04.448433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ddc24-c95c-4b3d-a373-0c5f010f3197 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.449110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ddc24-c95c-4b3d-a373-0c5f010f3197 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.452398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ddc24-c95c-4b3d-a373-0c5f010f3197 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.455955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 182/349] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:04.458471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:04.460096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.460602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.461491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.464684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:04.494085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:04.495244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:04.591456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 168/350] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:03:04.592798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:04.610350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:04.677353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 183/351] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:04.678294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:04.707943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb107-bc62-4a68-b210-38dbd3e2d463 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:04.746325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:04.747077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.747363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.747656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.759332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:04.768173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:04.769176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:04.770005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 169/352] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:04.770666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:04.773295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:04.845935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:04.879636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28dcf-6322-4b57-9069-bf57888ea41d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:04.891401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325584-98e9-43fe-86de-7f28e87c8bc6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:04.915766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ffdb-f9b7-4aa5-bf8e-3c542ff85d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:04.916856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ffdb-f9b7-4aa5-bf8e-3c542ff85d24 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.917156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ffdb-f9b7-4aa5-bf8e-3c542ff85d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.917421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ffdb-f9b7-4aa5-bf8e-3c542ff85d24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.927483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:04.928386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.928662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.928932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.929781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189ffdb-f9b7-4aa5-bf8e-3c542ff85d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:04.930158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:04.931657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:04.933689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 184/353] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:04 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:04.934531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2189ffdb-f9b7-4aa5-bf8e-3c542ff85d24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:04.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:04.939184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:04.939958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 170/354] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:04.942171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bddae0-5e84-482d-9209-8ec398d08873 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:04.942473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bddae0-5e84-482d-9209-8ec398d08873 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:04.942771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bddae0-5e84-482d-9209-8ec398d08873 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:04.944378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 171/355] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:04.945102 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bddae0-5e84-482d-9209-8ec398d08873 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.049559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.056581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea77cc7-30af-4a47-9331-3448e7f02c6b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.068450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bddae0-5e84-482d-9209-8ec398d08873 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.105991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.107820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.108353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.108454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.109173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:05.109723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:05.111013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.112340 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:05.117109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ce22-7896-42c5-b651-43774fac78b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.118248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ce22-7896-42c5-b651-43774fac78b1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.118728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ce22-7896-42c5-b651-43774fac78b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.119182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ce22-7896-42c5-b651-43774fac78b1 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.120536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470ce22-7896-42c5-b651-43774fac78b1 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:05.122242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470ce22-7896-42c5-b651-43774fac78b1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:05.123127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 185/356] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:05.124300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.125252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 172/357] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:04 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:05.131076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.131660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.132116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.152390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:05.154166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:05.236007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 173/358] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:05.236879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.250433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca138ef-5887-4e6a-b2b4-e87d6755143a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.294930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.295828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.296002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.296289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.297049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:05.297356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:05.298772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:05.309362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 186/359] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:05.310631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.337182 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.359340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4f4f7-5bbc-4f6b-8684-b666eb6a429f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.396510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.403343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.406262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.406262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.406262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.442481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5be5a-b9bf-44aa-9d43-c4a288851ac4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.480771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.481859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.481859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.482049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.483113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=536792d0280e402db36f3fcc605a2dbd) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:05.483399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:05.484851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b7c61-281d-4bee-9101-c24102ed2f09 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:05.490793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 187/360] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:05.492079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa33eba-848a-4a49-80d4-2e62517d7013 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.492362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa33eba-848a-4a49-80d4-2e62517d7013 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.492623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa33eba-848a-4a49-80d4-2e62517d7013 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.493820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:05.493952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 188/361] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:05.494694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa33eba-848a-4a49-80d4-2e62517d7013 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.495431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:05.612977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.634043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 174/362] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:05.634376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa33eba-848a-4a49-80d4-2e62517d7013 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.634942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.676289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.676966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba4a98-4b02-4e1a-889b-d7bfc5336cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.677922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba4a98-4b02-4e1a-889b-d7bfc5336cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.678472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba4a98-4b02-4e1a-889b-d7bfc5336cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.678472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba4a98-4b02-4e1a-889b-d7bfc5336cf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.692822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba4a98-4b02-4e1a-889b-d7bfc5336cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7fba4a98-4b02-4e1a-889b-d7bfc5336cf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:05.701159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:05.702467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:05.703484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:05.711882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 189/363] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:05.711882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.711882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.711882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.735558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:05.736763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:05.791465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc4efd-17e8-48ce-896e-e590e8e81b60 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.828392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 190/364] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:05.834741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.840130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:05.841392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:05.841906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:05.842407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] PATH_INFO: `/v3/users/7eed777e01bd48fc8c14e5a86af80e39` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:05.847538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] RBAC: Authorizing `identity:get_user(user_id=7eed777e01bd48fc8c14e5a86af80e39)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:05.849273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:05.851578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 175/365] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/users/7eed777e01bd48fc8c14e5a86af80e39 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:05.852587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:05.942320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.970105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713abef-c4c0-4fb4-a411-be5f3ef83d37 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:05.973816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:05.980539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099875ce-f4f1-4af3-a9b1-a543b94fabcd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.009255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.010143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.010534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.010850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.012072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.013704 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.016023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 191/366] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:06.016879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.023512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.024189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.024472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.024742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.025494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:06.025798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.027335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.038479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 176/367] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:06.039444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.156802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736484-5eeb-4b18-8159-6812754f7c36 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.167134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f48fee-d761-4775-8f9a-c17dd6e0e09f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.194601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.195311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.195621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.195971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.196816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] RBAC: Adding query filter params (role.id=a23a16fa358b491c876189b3d05e60c7, user.id=7eed777e01bd48fc8c14e5a86af80e39, scope.system=all) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:06.197129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.198796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.203584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.204382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.204662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.204930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.209095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:06.213787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 192/368] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/role_assignments?user.id=7eed777e01bd48fc8c14e5a86af80e39&scope.system=all&role.id=a23a16fa358b491c876189b3d05e60c7 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:06.214602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.222192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:06.230269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:06.231197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:06.232009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 177/369] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:06.235567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.283415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:06.340499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c636f-4c21-418c-a6f0-ce73d2d305df None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.383747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.386609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.387180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.387605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.389255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.390969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.393121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 193/370] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:06.394293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.475780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:06.518940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a7831-302b-4420-983d-1c48f9cc53a2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.530327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85e902-3515-4b68-86c2-53426707fa83 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.562160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.562889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.563173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.563438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.564194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:06.564498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.565952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.574911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65346f67-1c91-40e5-9367-f321c2aa05e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.575773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 194/371] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:06.575970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65346f67-1c91-40e5-9367-f321c2aa05e2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.576559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65346f67-1c91-40e5-9367-f321c2aa05e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.577174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65346f67-1c91-40e5-9367-f321c2aa05e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.582724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.687779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65346f67-1c91-40e5-9367-f321c2aa05e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.690063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65346f67-1c91-40e5-9367-f321c2aa05e2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.717734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:06.723630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cecd0-0889-4e29-8df1-749498e5f4ff None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:06.772597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:06.782004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:06.783464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.784077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.784633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.785663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:06.786168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:06.788511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:06.796757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 195/372] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:06.797879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.858934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 178/373] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:06.860429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252717-2216-422c-b37e-1cc45aaaa9ab None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:06.860790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252717-2216-422c-b37e-1cc45aaaa9ab None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:06.861064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252717-2216-422c-b37e-1cc45aaaa9ab None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:06.862436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 179/374] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:06.868893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252717-2216-422c-b37e-1cc45aaaa9ab None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:06.947794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e0fec-a9d6-47d8-92a4-93b17e531c6d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.005106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252717-2216-422c-b37e-1cc45aaaa9ab None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.008327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.009709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.009709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.009709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.036858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:07.055481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487f209-d4fc-4357-9ec9-be4d4c41d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.056427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487f209-d4fc-4357-9ec9-be4d4c41d611 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.056711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487f209-d4fc-4357-9ec9-be4d4c41d611 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.056986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487f209-d4fc-4357-9ec9-be4d4c41d611 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.058196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487f209-d4fc-4357-9ec9-be4d4c41d611 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.060159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487f209-d4fc-4357-9ec9-be4d4c41d611 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.063844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 180/375] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:07.065154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.065447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.065726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.086279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:07.087571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:07.088002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:07.096524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.098065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.173098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 181/376] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:07.174730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.227087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 196/377] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:07.232917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.300672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e90162-cdbd-493f-b314-da421973cc39 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.328800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:07.346438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:07.359570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b2b4d-1ba2-43dd-8c8c-ddf05e0215d1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.364107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.365291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.365776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.366251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.367755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.369584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.372064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 182/378] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:07.374521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.404019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.405047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.405419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.406550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.423368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:07.439174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:07.440109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:07.440939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 197/379] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:07.442242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.519588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c386c61-28f5-4971-9462-57abb93fddf0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.577850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.578657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.578990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.579265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] PATH_INFO: `/v3/projects/98caa12ca1b14e52b73e3cce22837778` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.581144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] RBAC: Authorizing `identity:get_project(project_id=98caa12ca1b14e52b73e3cce22837778)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.582933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.585932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 183/380] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/projects/98caa12ca1b14e52b73e3cce22837778 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:07.590483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.597115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e2024-7d19-4720-a9c6-c4fb4d7d1e79 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.641308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.642375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.642891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.643367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.644443 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:07.644958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.646629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.653118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 198/381] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:07.654390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.660437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:07.680902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:07.734104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e7d87-e430-4569-84da-548550a4bdd6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.783162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.783396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.783637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.783828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.784595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:07.784906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.787304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.799611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 184/382] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:07.807035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.826410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc754ad-7a86-49bf-abb8-420068fae995 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:07.895088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e1bae-64cd-4351-b61a-6b13017f3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:07.895984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e1bae-64cd-4351-b61a-6b13017f3715 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.896266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e1bae-64cd-4351-b61a-6b13017f3715 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.896541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e1bae-64cd-4351-b61a-6b13017f3715 None admin] PATH_INFO: `/v3/groups/138250482dfd4b40920b6c00be2e1ae0` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.905847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e1bae-64cd-4351-b61a-6b13017f3715 None admin] RBAC: Authorizing `identity:get_group(group_id=138250482dfd4b40920b6c00be2e1ae0)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:07.907392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e1bae-64cd-4351-b61a-6b13017f3715 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:07.910386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 199/383] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/groups/138250482dfd4b40920b6c00be2e1ae0 => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:07.915896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fcb8-b50a-4b9a-aeaf-5e0fbd6d85ed None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:07.917938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fcb8-b50a-4b9a-aeaf-5e0fbd6d85ed None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:07.918229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fcb8-b50a-4b9a-aeaf-5e0fbd6d85ed None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:07.921118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 200/384] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:07.921118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fcb8-b50a-4b9a-aeaf-5e0fbd6d85ed None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:07.990511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52b4b9-2d72-4bf5-b332-949ec6aad08f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.012794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:08.038029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f7900-44e4-4419-8eaa-0c0f3f99b80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.039047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:08.039287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f7900-44e4-4419-8eaa-0c0f3f99b80a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.039582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f7900-44e4-4419-8eaa-0c0f3f99b80a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.039859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f7900-44e4-4419-8eaa-0c0f3f99b80a None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.043036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f7900-44e4-4419-8eaa-0c0f3f99b80a None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:08.043036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f7900-44e4-4419-8eaa-0c0f3f99b80a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:08.044938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 185/385] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:08.050004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.050004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.050004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.069071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:08.070423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:08.070614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fcb8-b50a-4b9a-aeaf-5e0fbd6d85ed None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.114626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.116753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.117205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.117743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.118759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] RBAC: Adding query filter params (group.id=138250482dfd4b40920b6c00be2e1ae0, role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=98caa12ca1b14e52b73e3cce22837778) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:08.119253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:08.120955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:08.127500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 201/386] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:07 2022] GET /identity/v3/role_assignments?group.id=138250482dfd4b40920b6c00be2e1ae0&scope.project.id=98caa12ca1b14e52b73e3cce22837778&role.id=b10ab4271e1845e4aefab70d82a12def => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:08.128652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:08.165859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 186/387] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:08.208262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:08.325070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c00dc-4beb-48dd-b811-60f25771b55e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.375956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.376957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.377235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.377505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.378724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:08.380376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:08.382310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 202/388] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:08 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:08.386190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab4fc-1255-4051-8fba-a8e285465eae None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.389258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:08.403610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:08.411125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:08.482355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.483273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.483476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.483748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.506651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be9568-9859-4ff5-ab57-a7725bdaca07 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.519529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:08.527997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:08.528876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:08.530356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:08.538726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 187/389] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:08.544930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:08.568000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.569285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.569568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.569845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd/groups/a6a53caf87a14271875e94be7e37137a/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.574536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] RBAC: Authorizing `identity:create_grant(project_id=536792d0280e402db36f3fcc605a2dbd, group_id=a6a53caf87a14271875e94be7e37137a, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:08.575820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:08.600814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 203/390] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Jun 24 10:03:08 2022] PUT /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd/groups/a6a53caf87a14271875e94be7e37137a/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:08.703985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3966-5097-4c3e-b1da-25fcfdd58bcd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.747216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.747805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.748435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.749366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.749942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:08.750251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:08.751802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:08.762828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 188/391] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:08.768986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:08.872663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:08.902268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfb50e-82bd-4788-84d7-4cd99ddf5906 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:08.940898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:08.941783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:08.942067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:08.942344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:08.943545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:08.944999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:08.946646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 204/392] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:08 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:08.953068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:08.999128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:09.075278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112c270-3278-4eec-9efc-272f4dd97cce None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:09.117916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bca72-1120-4774-8983-3abb3b4f6f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:09.118822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bca72-1120-4774-8983-3abb3b4f6f8e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:09.119152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bca72-1120-4774-8983-3abb3b4f6f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:09.119428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bca72-1120-4774-8983-3abb3b4f6f8e None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:09.120630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bca72-1120-4774-8983-3abb3b4f6f8e None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:09.122095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bca72-1120-4774-8983-3abb3b4f6f8e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:09.124277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 189/393] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:08 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:09.129946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:09.252964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06135d8-923f-4f88-abc0-5fa96fd9d45e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:09.297268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359888e-e5a6-467a-919c-137ac8603a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:09.298510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359888e-e5a6-467a-919c-137ac8603a89 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:09.298991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359888e-e5a6-467a-919c-137ac8603a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:09.299434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359888e-e5a6-467a-919c-137ac8603a89 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd/groups/a6a53caf87a14271875e94be7e37137a/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:09.301815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359888e-e5a6-467a-919c-137ac8603a89 None admin] RBAC: Authorizing `identity:create_grant(project_id=536792d0280e402db36f3fcc605a2dbd, group_id=a6a53caf87a14271875e94be7e37137a, role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:09.303482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359888e-e5a6-467a-919c-137ac8603a89 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:09.322760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 205/394] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Jun 24 10:03:09 2022] PUT /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd/groups/a6a53caf87a14271875e94be7e37137a/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:09.555458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cdb2-f159-4eb0-9575-f2a221d1499e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:09.556016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cdb2-f159-4eb0-9575-f2a221d1499e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:09.556461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cdb2-f159-4eb0-9575-f2a221d1499e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:09.558391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 190/395] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:09.563696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:09.564312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:09.564783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:09.571255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:09.587438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:09.589258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:09.675201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 206/396] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:09.711164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433cdb2-f159-4eb0-9575-f2a221d1499e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:09.827141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433cdb2-f159-4eb0-9575-f2a221d1499e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:09.829676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:09.881362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:09.882182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:09.882460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:09.882871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:09.898969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:09.907019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:09.907881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:09.908654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 191/397] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:09.917454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.035950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92ddd-3c07-42d7-a433-ff771af31408 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:10.070734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:10.076364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:10.077876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.078350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.078838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.079822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:10.080317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:10.082029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cd879-5234-4bbc-939b-a397ab09d7dc None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:10.093892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 207/398] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:10.099159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.261544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a7cc9-b974-4ca5-9a8d-05d9063a1dfa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:10.303983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:10.304860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.305142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.305416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.311515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:10.315347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:10.315483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36b40-05c5-46f5-a5c9-b12957ce595a None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.315483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36b40-05c5-46f5-a5c9-b12957ce595a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.315483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36b40-05c5-46f5-a5c9-b12957ce595a None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.315678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 192/399] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:10 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:10.316727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 208/400] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:10.320700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.322822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.323307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.324070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.346074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:10.347977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:10.364463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:10.446728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 209/401] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:10.471401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e050103-0304-4c4a-8b30-0d06caf422ad None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:10.476639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.548408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:10.549306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.549587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.549858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.551119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:10.552581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:10.558956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 193/402] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:10 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:10.563708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.601508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01074be0-f6aa-4912-9766-c81c5f2e3301 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:10.629544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:10.651501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a877cf-7b26-48c4-8197-c08d67f9036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:10.652343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a877cf-7b26-48c4-8197-c08d67f9036e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.652628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a877cf-7b26-48c4-8197-c08d67f9036e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.652913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a877cf-7b26-48c4-8197-c08d67f9036e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.669074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a877cf-7b26-48c4-8197-c08d67f9036e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-01a877cf-7b26-48c4-8197-c08d67f9036e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:10.685684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:10.686612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:10.691073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 210/403] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:10.701022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06d5ef-4828-48a7-ae72-174b67e526bc None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.701316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06d5ef-4828-48a7-ae72-174b67e526bc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.701601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06d5ef-4828-48a7-ae72-174b67e526bc None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.703271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 211/404] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:10.703983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06d5ef-4828-48a7-ae72-174b67e526bc None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.725765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:10.730177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0514a5-2e8c-486e-9c8c-13556ec6598d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:10.766554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb3b95-89c5-4bdb-9108-b1f76e8bd375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:10.767680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3b95-89c5-4bdb-9108-b1f76e8bd375 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.767960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3b95-89c5-4bdb-9108-b1f76e8bd375 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.768509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3b95-89c5-4bdb-9108-b1f76e8bd375 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/groups/a6a53caf87a14271875e94be7e37137a/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.770581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb3b95-89c5-4bdb-9108-b1f76e8bd375 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce6397fd8d4748b48261a3f3af2fc42a, group_id=a6a53caf87a14271875e94be7e37137a, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:10.772834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb3b95-89c5-4bdb-9108-b1f76e8bd375 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:10.791878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 194/405] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Jun 24 10:03:10 2022] PUT /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/groups/a6a53caf87a14271875e94be7e37137a/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:10.797447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.797758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.802733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.830783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:10.832030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:10.858137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06d5ef-4828-48a7-ae72-174b67e526bc None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:10.932085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:10.932927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:10.933087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:10.933539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:10.934319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:10.936366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:10.940262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:10.960026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 212/406] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:10.964367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:10.987661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 195/407] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:11.013360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.020198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.076245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a1e2-9112-4b16-8fe8-8bd8b82aba72 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.115094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.115956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.116310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.116595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.119040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:11.120554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:11.122980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 213/408] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:10 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:11.125110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.129472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.141160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62f31-94c6-47eb-8924-6c57c97abf14 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.179997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.181281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.181777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.182304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.193821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:11.202399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:11.203474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:11.204377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:11.206576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 196/409] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:11.212422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.303583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40a4f6-004b-448b-8ffb-7f3316acd790 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.320201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ededde-5c6b-4b53-8255-538120544ab8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.325075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.359893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.360826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.361277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.361692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.362587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:11.363059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:11.364616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:11.378977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 197/410] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:11.380163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.386277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.387250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.387537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.387913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.389201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:11.390728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:11.398379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.399219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 214/411] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:11.405485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.493198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea04a6e-2487-4c35-8c18-084a255a666c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.531215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ebe8-a746-44ee-af14-e10509e716bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.532577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ebe8-a746-44ee-af14-e10509e716bd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.533001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ebe8-a746-44ee-af14-e10509e716bd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.533446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ebe8-a746-44ee-af14-e10509e716bd None admin] PATH_INFO: `/v3/groups/138250482dfd4b40920b6c00be2e1ae0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.534928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065ebe8-a746-44ee-af14-e10509e716bd None admin] RBAC: Authorizing `identity:get_group(group_id=138250482dfd4b40920b6c00be2e1ae0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:11.536654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065ebe8-a746-44ee-af14-e10509e716bd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:11.538686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 198/412] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/groups/138250482dfd4b40920b6c00be2e1ae0 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:11.540023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348563d-dfd5-4cc0-8591-1802cd54e131 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.540470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348563d-dfd5-4cc0-8591-1802cd54e131 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.540882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348563d-dfd5-4cc0-8591-1802cd54e131 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.542621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 199/413] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:11.548350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.548808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.549244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.567849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:11.569390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:11.589697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87717059-5c2b-4ff3-9ae1-0a3b7b9b5fd3 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.631916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.633153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.633589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.634001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/groups/a6a53caf87a14271875e94be7e37137a/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.636501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] RBAC: Authorizing `identity:create_grant(project_id=ce6397fd8d4748b48261a3f3af2fc42a, group_id=a6a53caf87a14271875e94be7e37137a, role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:11.638326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:11.653176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 215/414] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Jun 24 10:03:11 2022] PUT /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a/groups/a6a53caf87a14271875e94be7e37137a/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:11.654817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.663039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 200/415] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:11.682966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.696642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.712212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.779607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a6fbe-d40b-4563-afd9-6f6704d0a80e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.801050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67ec75-3c19-436d-af4c-0a5cf1967e55 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.818583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.819762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.820223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.820648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] PATH_INFO: `/v3/projects/98caa12ca1b14e52b73e3cce22837778` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.822375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] RBAC: Authorizing `identity:get_project(project_id=98caa12ca1b14e52b73e3cce22837778)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:11.824080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:11.826622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 216/416] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/projects/98caa12ca1b14e52b73e3cce22837778 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:11.832924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.841198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:11.842346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:11.842793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:11.843174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:11.855996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:11.864890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:11.865872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:11.866796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 201/417] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:11.872909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:11.950758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.960589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fb252-4aea-4a1a-9ada-b6a98e46dce0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:11.974174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:11.988928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339347-e425-4234-8dbd-c5c5732f41ae None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:12.007279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.008176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.008457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.008726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] PATH_INFO: `/v3/projects/98caa12ca1b14e52b73e3cce22837778/groups/138250482dfd4b40920b6c00be2e1ae0/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.012218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=98caa12ca1b14e52b73e3cce22837778, group_id=138250482dfd4b40920b6c00be2e1ae0, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:12.014141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:12.027353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.028244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.028538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.029044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.029934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:12.030258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:12.031658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba829a-2ae6-40ff-9326-710640547534 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:12.035629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 217/418] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Jun 24 10:03:11 2022] PUT /identity/v3/projects/98caa12ca1b14e52b73e3cce22837778/groups/138250482dfd4b40920b6c00be2e1ae0/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:12.044220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 202/419] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:12.051194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:12.199232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c45cb2-88b8-4884-b6dd-8b8529d1a8f5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:12.252999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.253989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.254449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.254923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.290749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08d7d6-a46b-457b-9b3c-8135b2bdc83d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.291303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08d7d6-a46b-457b-9b3c-8135b2bdc83d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.291303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08d7d6-a46b-457b-9b3c-8135b2bdc83d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.292927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 203/420] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:12.298813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.299097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.299372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.323218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:12.328185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:12.329392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:12.393326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:12.395338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:12.412086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 204/421] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:12.438365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:12.508078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:12.525165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 218/422] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:12.534751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:12.589516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bdfb7-23e8-489d-b9f7-fe2b2a1ade65 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:12.627861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:12.644096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.644918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.645202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.645474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.657412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb89331-00d2-430f-a936-00cb15b1ed5e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:12.664494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:12.672589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:12.673479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:12.674250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 205/423] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:12.683959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:12.725401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.726293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.726580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.726887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.731424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:12.733206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:12.735179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 219/424] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:12.741052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:12.831064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bddb7c-95ed-49fe-a861-3fc48ea4488b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:12.855656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:12.871365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.873071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.874795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.877412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.878361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:12.878851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:12.880393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:12.890750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 206/425] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:12.899746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:12.936252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7b5f-33ca-4ae1-b664-867463193184 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:12.976659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:12.977543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:12.977823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:12.978095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:12.979647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:12.981198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:12.983747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 220/426] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:12.989414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:13.004832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:13.100331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd72c5-45a0-4195-83c8-fb90f26a8f88 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:13.105890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad71f6-4238-4aa0-b078-f6ba73b8995e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:13.137681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:13.138349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.138993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.139278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.140119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=536792d0280e402db36f3fcc605a2dbd) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:13.140430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:13.141882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:13.148717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 221/427] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:13.184457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650a088-7e72-4999-82e9-2fd2d8265125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:13.185675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650a088-7e72-4999-82e9-2fd2d8265125 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.186188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650a088-7e72-4999-82e9-2fd2d8265125 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.186672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650a088-7e72-4999-82e9-2fd2d8265125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.282183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:13.309833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650a088-7e72-4999-82e9-2fd2d8265125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:13.312189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650a088-7e72-4999-82e9-2fd2d8265125 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:13.444258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 207/428] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:13.450027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:13.531663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:13.563230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e23ae-5c28-44f7-bdbe-d720ae0e6ac8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:13.602662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14838fa6-e693-4265-ac1f-cbcecdc259f7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.603331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14838fa6-e693-4265-ac1f-cbcecdc259f7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.603798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14838fa6-e693-4265-ac1f-cbcecdc259f7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 208/429] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:13.608075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:13.608963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.609244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.609525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] PATH_INFO: `/v3/projects/536792d0280e402db36f3fcc605a2dbd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.610912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] RBAC: Authorizing `identity:get_project(project_id=536792d0280e402db36f3fcc605a2dbd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:13.612067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.612358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.612460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:13.612639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.614369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 222/430] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:13 2022] GET /identity/v3/projects/536792d0280e402db36f3fcc605a2dbd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:13.619498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:13.638168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:13.639826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:13.746905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5affa-17be-418f-a5ea-5959e1f50248 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:13.761430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 209/431] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:13.789949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:13.791059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:13.791960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.792575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.792915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.795105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:13.796588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:13.798624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 223/432] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:13 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:13.804066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:13.812548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:13.823111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:13.944711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c7b56-3780-4810-9919-cb35bf74b444 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:13.956172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e459-1750-4103-bc64-4594df21a772 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:13.982516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f5e15-2da5-4b78-976c-600829bd3191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:13.983848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5e15-2da5-4b78-976c-600829bd3191 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:13.984335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5e15-2da5-4b78-976c-600829bd3191 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:13.984822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5e15-2da5-4b78-976c-600829bd3191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:13.995793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f5e15-2da5-4b78-976c-600829bd3191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:13.999375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.000083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.000370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.000650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.001499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=536792d0280e402db36f3fcc605a2dbd) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:14.001819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.003300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-298f5e15-2da5-4b78-976c-600829bd3191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:14.005932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:14.006832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 10:03:14.007630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:14.009829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 224/433] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:13 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=536792d0280e402db36f3fcc605a2dbd&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:14.010670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 210/434] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:14.022779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.030884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596df204-57a6-4714-84ff-a5e26b464639 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.031614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596df204-57a6-4714-84ff-a5e26b464639 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.032113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596df204-57a6-4714-84ff-a5e26b464639 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.035374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 211/435] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:14.040426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.041088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.041591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.068338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:14.070812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:14.098901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.135185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e67af-5db5-4dc3-81dc-e87f833e39bd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.167205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 212/436] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:14.181727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.182424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.182766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.183080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.183839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] RBAC: Adding query filter params (name=member) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:14.184147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.185564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.191944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.195183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.197530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 225/437] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:14.203452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.327940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3207-a05c-4ea4-b8d4-f02601694258 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.339849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf2e97-325c-417c-b944-d3ec68806081 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.370491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.371451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.371800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.372077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.380150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.385329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.386008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.386307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.387811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.392476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:14.400758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:14.401660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:14.402447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:14.403787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 213/438] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:14.410517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.475585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.477114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.498332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.542818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83239f4-83d5-436a-a15b-7b02bf5442ff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.579424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.580016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.580298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.580579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.581336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:14.581649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.583087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.585414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 226/439] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 10:03:14.590936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.598308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 214/440] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:14.599232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.622511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.696835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2454d5e-5a48-46e8-b123-c31a078789cf None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.709629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e835b97-cbd0-4831-81b7-a9625017c0bc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.733400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.734269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.734549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.734855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.736047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.737528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.738847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.739541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 227/441] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:14.744988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.753729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77b619-218b-40ec-b7f6-ea42bd708d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.754389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77b619-218b-40ec-b7f6-ea42bd708d5e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.754673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77b619-218b-40ec-b7f6-ea42bd708d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.754981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77b619-218b-40ec-b7f6-ea42bd708d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.823370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77b619-218b-40ec-b7f6-ea42bd708d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.824845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77b619-218b-40ec-b7f6-ea42bd708d5e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.844291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.847642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede5426-9605-4476-8ac7-596c6018de84 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:14.882228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4cf42-5904-4305-b0d8-907685633040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:14.883155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4cf42-5904-4305-b0d8-907685633040 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.883425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4cf42-5904-4305-b0d8-907685633040 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.883690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4cf42-5904-4305-b0d8-907685633040 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.884913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4cf42-5904-4305-b0d8-907685633040 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:14.886389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4cf42-5904-4305-b0d8-907685633040 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:14.888028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 228/442] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:14.889753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3ca2-75ef-447d-adc5-25d7152ada96 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.890033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3ca2-75ef-447d-adc5-25d7152ada96 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.890303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3ca2-75ef-447d-adc5-25d7152ada96 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.895422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 229/443] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:14.896175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3ca2-75ef-447d-adc5-25d7152ada96 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:14.929577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 215/444] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:14.930882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:14.931163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:14.931412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:14.946980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:14.948173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:14.970472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:14.997192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3ca2-75ef-447d-adc5-25d7152ada96 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.016037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 216/445] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:15.016942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.040009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.040678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.040988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.041267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.042132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=0f531bdb05494dcf9bf516de1fb46554, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:15.042444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.043934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.050055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 230/446] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:14 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=0f531bdb05494dcf9bf516de1fb46554 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:15.050935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.086485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:15.147060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1dc94-91ac-4e53-ac06-e2d8c1f1646c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.172391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fe5c3-13c9-42ce-af13-1c27db8e47f1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.186209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.187125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.187432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.187682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] PATH_INFO: `/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.189270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] RBAC: Authorizing `identity:get_project(project_id=ce6397fd8d4748b48261a3f3af2fc42a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.190783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.192709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 217/447] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/projects/ce6397fd8d4748b48261a3f3af2fc42a => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:15.198321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.211634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.212434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.212713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.212979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.225677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:15.225850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:15.234246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:15.235329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:15.236097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:15.236704 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 231/448] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:15.241901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.319533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3209a0-7331-4e7b-8d09-d4a109657c59 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.349374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:15.365031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9675832-bc1f-497e-b94d-1bfd90641064 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.369082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.370491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.370989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.371413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] PATH_INFO: `/v3/groups/a6a53caf87a14271875e94be7e37137a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.373497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] RBAC: Authorizing `identity:get_group(group_id=a6a53caf87a14271875e94be7e37137a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.376634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.380088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 218/449] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/groups/a6a53caf87a14271875e94be7e37137a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:15.386145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.406896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.408148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.408148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.408148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.408785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:15.409093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.410481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.422090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 232/450] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:15.427918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.489220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:15.498638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317a48-9c09-420e-a944-56dfbbc8eea9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.535067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.535726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.536008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.536280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.537103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] RBAC: Adding query filter params (group.id=a6a53caf87a14271875e94be7e37137a, role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:15.537426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.538911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.558737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 219/451] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/role_assignments?group.id=a6a53caf87a14271875e94be7e37137a&scope.project.id=ce6397fd8d4748b48261a3f3af2fc42a&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:15.558964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d8a1c-b709-4703-bcd0-7c39ddcf5e4a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.585050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.585721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.586011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.586312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.613721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:15.661520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.663039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.771349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 233/452] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:15.777067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:15.892782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6fbd-8804-4445-8282-7063bdff7ea5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:15.927961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:15.928746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:15.929028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:15.929299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] PATH_INFO: `/v3/projects/98caa12ca1b14e52b73e3cce22837778` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:15.930525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] RBAC: Authorizing `identity:get_project(project_id=98caa12ca1b14e52b73e3cce22837778)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:15.932046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:15.933772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 220/453] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/projects/98caa12ca1b14e52b73e3cce22837778 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:15.940234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:16.007134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:16.063305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5701bf-9058-4407-a585-09806d0f6dc3 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:16.099060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a8362-9e25-4c79-91dc-63d057ffbe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:16.099907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a8362-9e25-4c79-91dc-63d057ffbe3d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:16.100185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a8362-9e25-4c79-91dc-63d057ffbe3d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:16.100457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a8362-9e25-4c79-91dc-63d057ffbe3d None admin] PATH_INFO: `/v3/groups/138250482dfd4b40920b6c00be2e1ae0` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:16.101684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a8362-9e25-4c79-91dc-63d057ffbe3d None admin] RBAC: Authorizing `identity:get_group(group_id=138250482dfd4b40920b6c00be2e1ae0)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:16.103171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a8362-9e25-4c79-91dc-63d057ffbe3d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:16.105107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 234/454] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Jun 24 10:03:15 2022] GET /identity/v3/groups/138250482dfd4b40920b6c00be2e1ae0 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:16.111998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:16.140026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:16.218204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67174d0c-114f-4716-84e9-033a427aa8dc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:16.258268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:16.258933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:16.259206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:16.259475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:16.260295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] RBAC: Adding query filter params (group.id=138250482dfd4b40920b6c00be2e1ae0, role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=98caa12ca1b14e52b73e3cce22837778) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:16.260606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:16.262055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871edd7b-decf-435f-855d-1ea4730b0ccb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:16.268151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 221/455] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Jun 24 10:03:16 2022] GET /identity/v3/role_assignments?group.id=138250482dfd4b40920b6c00be2e1ae0&scope.project.id=98caa12ca1b14e52b73e3cce22837778&role.id=b10ab4271e1845e4aefab70d82a12def => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:18.719586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892fdcc-74ce-4543-8e7c-5b24152de26f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:18.720035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892fdcc-74ce-4543-8e7c-5b24152de26f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:18.720138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892fdcc-74ce-4543-8e7c-5b24152de26f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:18.721660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 235/456] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:18.726663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7878f8-5dac-4ad7-b493-fea8e5c3128d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:18.727446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7878f8-5dac-4ad7-b493-fea8e5c3128d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:18.727867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7878f8-5dac-4ad7-b493-fea8e5c3128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:18.756063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ad7878f8-5dac-4ad7-b493-fea8e5c3128d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:18.757521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad7878f8-5dac-4ad7-b493-fea8e5c3128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:18.852426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 222/457] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:18.898073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892fdcc-74ce-4543-8e7c-5b24152de26f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.048459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892fdcc-74ce-4543-8e7c-5b24152de26f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.081462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:19.098171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.099024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.099306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.099457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfc371-01cc-452a-8e5c-00a450e03e2d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.099595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.100036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfc371-01cc-452a-8e5c-00a450e03e2d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.100623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfc371-01cc-452a-8e5c-00a450e03e2d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.102376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 223/458] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:19.108208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.108492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.108769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.115825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.117311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:19.134095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:19.135324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:19.153652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:19.154597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:19.155602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:19.155602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:19.155602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:19.155602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:19.156943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 236/459] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:19.164107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.175064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:19.249666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 224/460] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:19.283858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.330191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b907ff8-15c7-4ec9-8f7f-d75b31928697 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.367357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fa012-931f-4abe-9919-696682efc190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.368031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fa012-931f-4abe-9919-696682efc190 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.368312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fa012-931f-4abe-9919-696682efc190 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.368586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fa012-931f-4abe-9919-696682efc190 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.369302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fa012-931f-4abe-9919-696682efc190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:19.369611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fa012-931f-4abe-9919-696682efc190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.371057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fa012-931f-4abe-9919-696682efc190 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:19.383756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 237/461] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:19.385001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588187-9e34-4914-8196-94b231c176d5 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.385288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588187-9e34-4914-8196-94b231c176d5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.385564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588187-9e34-4914-8196-94b231c176d5 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.390752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 238/462] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:19.390752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180585-724a-45b1-975d-6f4248b48ca7 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.390752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180585-724a-45b1-975d-6f4248b48ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.390752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180585-724a-45b1-975d-6f4248b48ca7 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.390752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 239/463] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:19.395405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a42187-552e-490f-ae8c-0ba360c0c97b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.395696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a42187-552e-490f-ae8c-0ba360c0c97b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.395979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a42187-552e-490f-ae8c-0ba360c0c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.401078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d48f3b-8131-4470-9141-0e8429bbcc22 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.412415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-24a42187-552e-490f-ae8c-0ba360c0c97b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:19.413598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-24a42187-552e-490f-ae8c-0ba360c0c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:19.438376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.439320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.439598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.439882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.440993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:19.450198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.451768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:19.460686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:19.472069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:19.472991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:19.473875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:19.473875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:19.473875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:19.473875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:19.473875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 225/464] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:19.474508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.498357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 240/465] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:19.498357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.498357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.498357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.513395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:19.515285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:19.589160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 241/466] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:19.589578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2d9cf-4d96-4339-b282-f5272c25d701 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.590609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.630582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.631396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.631758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.632045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.632763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.634248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:19.654492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 226/467] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:03:19 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:19.654492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.722987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:19.731850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:19.733683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3e18e-51e7-470b-8cc0-7911f4d47f77 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.788017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569300a-fe74-4f12-8673-f1e7cd632cde None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.788388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.789022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.789304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.789854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.791667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:19.791983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.793453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:19.814907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 242/468] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:19.815890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.832432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.833610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.833894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.834172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.846791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.848213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:19.870449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:19.871372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:19.872208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:19.872208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:19.872208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:19.872208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:19.872208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 227/469] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:19.877540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:19.944897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc461be-6665-4661-ab58-34eba5330efb None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:19.981976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:19.982919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:19.983361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:19.983641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:19.997961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:19.999720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.003579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c8c7b-8966-4d50-a2b1-59ebbca77881 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.012851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:20.019952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:20.020824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:20.021654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:20.021654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:20.021654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:20.021654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:20.026442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 243/470] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:20.027339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:20.038843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:20.052224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.052910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.053192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.053471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.054184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:20.055741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.082781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 228/471] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:03:19 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:20.084087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:20.203524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9860be-bbd4-4d3e-866a-07a633c23d6b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762426-0e48-49e6-93a2-cba909ed1aa0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.257146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.257832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.258118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.258395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.259169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:20.259514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:20.260884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.261066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56605da-16be-464d-b317-647ff5562649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.261705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56605da-16be-464d-b317-647ff5562649 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.261985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56605da-16be-464d-b317-647ff5562649 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.262261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56605da-16be-464d-b317-647ff5562649 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.263035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56605da-16be-464d-b317-647ff5562649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:20.263339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56605da-16be-464d-b317-647ff5562649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:20.265121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56605da-16be-464d-b317-647ff5562649 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.278840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 244/472] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:20.279924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:20.286638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 229/473] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:20.287848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2956a-1fbd-4815-b263-bc6963007213 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.288138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2956a-1fbd-4815-b263-bc6963007213 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.288451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2956a-1fbd-4815-b263-bc6963007213 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.289814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 230/474] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:20.295738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2956a-1fbd-4815-b263-bc6963007213 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:20.328778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:20.358974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:20.402694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb13737-2155-4f4f-8783-1660ad3bc065 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.430146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2956a-1fbd-4815-b263-bc6963007213 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.440907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86899112-ea8b-4dfb-affc-720f41a0dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.441672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899112-ea8b-4dfb-affc-720f41a0dc89 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.442319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899112-ea8b-4dfb-affc-720f41a0dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.442319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899112-ea8b-4dfb-affc-720f41a0dc89 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.443107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86899112-ea8b-4dfb-affc-720f41a0dc89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:20.444732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86899112-ea8b-4dfb-affc-720f41a0dc89 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.473118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 245/475] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:03:20 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:20.478769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.478769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.478769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.481707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.485332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.485646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.485925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.487562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:20.492092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.514068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:20.515348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:20.541438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 231/476] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:03:20 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:20.614045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 246/477] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:20.634112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:20.746080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:20.766424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca115f6c-0b55-4e8a-a15c-14e5a0eda571 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.800958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.801632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.801910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.802189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:20.803006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:20.804388 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:20.811887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 10:03:20.811887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 10:03:20.811887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [parameters: {'id': '541e2751041647a3bca32448fa724e7a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 10:03:20.811887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jun 24 10:03:20.829253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:03:20.830186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 10:03:20.831135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 10:03:20.831962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:03:20.832820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:03:20.833601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'id': '541e2751041647a3bca32448fa724e7a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:20.834356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 10:03:20.836858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:20.838792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 232/478] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Jun 24 10:03:20 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:20.838937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:20.903172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:20.953518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73166742-931a-4d55-a811-c98d4b61e9d5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:20.991382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bd2a3-3970-4062-8f50-fb621e3a154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:20.992214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bd2a3-3970-4062-8f50-fb621e3a154d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:20.992498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bd2a3-3970-4062-8f50-fb621e3a154d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:20.992774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bd2a3-3970-4062-8f50-fb621e3a154d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:21.007295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bd2a3-3970-4062-8f50-fb621e3a154d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-0f0bd2a3-3970-4062-8f50-fb621e3a154d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:21.012766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:21.016031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:21.017957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 247/479] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:21.027530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:21.186744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2944e3-4f13-4331-ade9-8098a53a7d8b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:21.222582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:21.223277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:21.223555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:21.223822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:21.224570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:21.224871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:21.226247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3168a-b82c-417d-a1aa-5c633e270515 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:21.235053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 233/480] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:21.969738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71e48-4282-4ba1-a9b7-c746bfde9c2f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:21.970250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71e48-4282-4ba1-a9b7-c746bfde9c2f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:21.970467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71e48-4282-4ba1-a9b7-c746bfde9c2f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:21.972225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 248/481] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:21.977502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:21.977925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:21.978101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:21.994833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:22.003679 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:22.004879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:22.092342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 234/482] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:22.136586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71e48-4282-4ba1-a9b7-c746bfde9c2f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:22.272440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71e48-4282-4ba1-a9b7-c746bfde9c2f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:22.288908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:22.316773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91153381-8b73-4eee-8530-f7b4a040195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:22.317593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153381-8b73-4eee-8530-f7b4a040195f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:22.317875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153381-8b73-4eee-8530-f7b4a040195f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:22.318146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153381-8b73-4eee-8530-f7b4a040195f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:22.335291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91153381-8b73-4eee-8530-f7b4a040195f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-91153381-8b73-4eee-8530-f7b4a040195f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:22.342966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:22.343817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:22.344602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 249/483] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:22.350067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:22.519815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b3f1-0ec4-4013-bb8a-0af321a035e2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:22.553322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4bcc2-ffce-4247-b4bb-5f728b7352e9 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:22.553627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4bcc2-ffce-4247-b4bb-5f728b7352e9 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:22.553843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4bcc2-ffce-4247-b4bb-5f728b7352e9 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:22.556062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 250/484] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:22.561228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:22.561674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:22.561968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:22.580017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:22.580696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:22.580977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:22.581253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:22.581395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:22.582003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:22.582306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:22.582910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:22.583707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:22.595302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:22.602260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 235/485] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:22.607825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:22.693683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 251/486] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:22.731885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:22.767501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5af5-447b-4642-ad43-6082eceb63ab None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:22.833172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:22.833881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:22.834159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:22.834429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:22.854105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:22.855979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:22.881984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:22.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:22.884083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:22.884083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:22.884083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:22.884083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:22.884083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 236/487] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:22.894037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:22.903479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4bef7-dae4-401c-8e14-302ceac2edd2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:22.905394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:22.960790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:22.961614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:22.961888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:22.962156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:22.983928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:22.991675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:22.993087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:22.994387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 252/488] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:23.001757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.011060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.091871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476492d8-622e-4b36-aeaf-21377aabb911 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.151427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.152112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.152403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.152679 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.153422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:23.153728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:23.155510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:23.184873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 237/489] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:23.186169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.191449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01a366-a67d-4f12-9ac6-5b6ea6f582df None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.242680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.243418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.243699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.243976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.244829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:23.245144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:23.246543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39d6b4-eb82-4e20-af2e-49e085db7a1d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:23.255846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 253/490] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:23.261710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c0559-9dc6-4580-9faa-7f4864fe522d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.262008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c0559-9dc6-4580-9faa-7f4864fe522d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.262294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c0559-9dc6-4580-9faa-7f4864fe522d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.266791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 254/491] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:23.267616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c0559-9dc6-4580-9faa-7f4864fe522d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.293148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6065b14-924b-4418-aa30-06bec701f529 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.296390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.304599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.331218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.332002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.332427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.332579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.340492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:23.342141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-edc0ef54-ceed-4876-adb9-85ac8c7e30f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:23.362434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:23.363460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:23.364329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:23.365680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 238/492] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:23.365680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed76f3-ce12-42f5-a319-bf793aad96fa None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.371198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed76f3-ce12-42f5-a319-bf793aad96fa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.371198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed76f3-ce12-42f5-a319-bf793aad96fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.384984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-29ed76f3-ce12-42f5-a319-bf793aad96fa None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:23.386288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-29ed76f3-ce12-42f5-a319-bf793aad96fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:23.395888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c0559-9dc6-4580-9faa-7f4864fe522d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.433232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.434026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.434322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.434586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.446382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:23.447880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:23.458161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 239/493] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:23.459721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b89af-8d3e-4325-862f-c0366651c5e0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.460143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b89af-8d3e-4325-862f-c0366651c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.460548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b89af-8d3e-4325-862f-c0366651c5e0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.461969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 240/494] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:23.462801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b89af-8d3e-4325-862f-c0366651c5e0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-713d5598-ce4a-4375-8a0e-e3268d5014b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:23.470026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:23.470949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:23.471793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:23.471793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:23.471793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:23.471793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:23.471793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 255/495] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:23.476697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.476697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.476697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.493295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:23.494494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:23.563905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 256/496] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:23.564765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.581039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.593802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.599412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b89af-8d3e-4325-862f-c0366651c5e0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.636452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.637229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.637516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.637786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.638880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:23.639235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:23.640902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:23.677083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bd56e-33c0-4011-b4c2-546164303233 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.684745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 241/497] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:23.684745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.719834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.720789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.721228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.721506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.722248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:23.722558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:23.724745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:23.736915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 257/498] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:23.737570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.799504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67fc87-92ac-4b09-9ca8-7a3a4ad7e058 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.804516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.837024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.837846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.838128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.838404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.842012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d4770-553e-4487-8ca5-e8896015188d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:23.848831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:23.856836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:23.857926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:23.858697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 242/499] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:23.859373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.881027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:23.883819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:23.884991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:23.885281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:23.885575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:23.896404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:23.904211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:23.905124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 258/500] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:23.910782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:23.975293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ab411-4835-486d-8cca-b920387e11d0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.009772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.010416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.010697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.010998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.011985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cdf73-d4c0-4875-9965-5e33b9d1baac None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.038294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.047118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.047924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.048204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.048482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] PATH_INFO: `/v3/users/glance` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.055981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.057452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:24.072611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:24.073738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 259/501] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:24.079631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.088417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.089908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.108904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.172970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa729c-cb0f-433b-8017-68c01fd35bda None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.192109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 243/502] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:24.193176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.210781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.211431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.211720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.212037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.212777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:24.213087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.214488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58738b-c803-4cff-bdd7-769ba51f7e16 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.222565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 260/503] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:24.223813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69b7ea-21b6-4b66-98dd-3de382ca8851 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.224102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69b7ea-21b6-4b66-98dd-3de382ca8851 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.224382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69b7ea-21b6-4b66-98dd-3de382ca8851 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.225626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 261/504] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:24.226311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69b7ea-21b6-4b66-98dd-3de382ca8851 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.277088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.300538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f19baf-29dc-4083-a227-475438e26dad None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.317554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.330960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69b7ea-21b6-4b66-98dd-3de382ca8851 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.334335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.335040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.335323 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.335597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.336345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:24.336652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.338092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.345980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 244/505] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:24.346792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.369316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.369930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.370218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.370497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.371383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:24.371695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.373225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.410104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 262/506] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:24.410957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.449733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40502a-e047-47f5-a530-cca41f7abf1d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.485442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.486237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.486526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.486831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.497542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.499094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.515566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3957-3c45-4a74-a0b7-9baff20eb37d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2d5e6342-9f25-4251-9525-bfb81e91223e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:24.516065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:24.517014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:24.518069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:24.518069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:24.518069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:24.518069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:24.518069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 245/507] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:24.518723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.518816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.519052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.524592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.535620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:24.536818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:24.553368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.554160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.554442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.554798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.563094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.564876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.566394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:24.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:24.584468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:24.585570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:24.585570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:24.585570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:24.585570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:24.585570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 263/508] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:24.585570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.607320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 246/509] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:24.608154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.700922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27c461-c98f-4e37-adfc-6770d0a81daf None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.708082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b614c-9619-43b0-a740-cfb8a45aa6fb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.736204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e13252-e478-441b-a883-cc85958d01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.737019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e13252-e478-441b-a883-cc85958d01df None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.737303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e13252-e478-441b-a883-cc85958d01df None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.737596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e13252-e478-441b-a883-cc85958d01df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.743002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.743652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.743934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.744207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.749798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e13252-e478-441b-a883-cc85958d01df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.751331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e13252-e478-441b-a883-cc85958d01df None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-04e13252-e478-441b-a883-cc85958d01df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:24.769467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:24.770439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:24.771487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:24.771487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:24.771487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:24.771487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:24.771487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 264/510] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:24.771487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e13252-e478-441b-a883-cc85958d01df None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.776580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.784950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.815673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.817162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.871984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e13252-e478-441b-a883-cc85958d01df None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:24.907503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:24.908171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:24.908456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:24.908731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:24.909461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:24.909767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:24.911203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:24.918082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 247/511] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:24.918948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.928411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 265/512] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:24.929235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:24.984104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:24.997749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.022865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600bc3c-3f91-4c10-82f6-78617e65e416 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.029614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61baa56-5ee1-44b1-9e7e-c576ce431f1d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.056155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.056827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.057106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.057382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.058109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:25.058415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.059858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.063843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.064639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.064919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.065190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.071175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 248/513] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:25.071979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.076149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.077645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:25.095086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:25.095916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:25.096756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:25.096756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:25.096756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:25.096756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:25.098842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 266/514] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:25.099823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.172330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8655852-051b-4d0a-8b82-423c1663e2b8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.193902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.199133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7254743-95b1-40db-bf1a-db076ccad255 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.206012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.206676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.206984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.207260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.207983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:25.208289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.209677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.221473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.221606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 249/515] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:25.222300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.238070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.238846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.239131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.239408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.249446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.250931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:25.268021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:25.269058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:25.270118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:25.270118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:25.270118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:25.270118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:25.270118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 267/516] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:25.270118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.325042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dca97-73ea-4230-b1c1-396ef22fc645 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.359707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.360522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.360800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.361072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.367551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.368834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fac64e-e8d8-4d70-ae41-8b735386b7ee None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.369005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:25.383272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:25.384305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:25.385124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 250/517] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:25.385601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.409464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.410271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.410556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.410868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.418102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.419700 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.430779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:25.434626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:25.435579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:25.436453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 268/518] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:25.437000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.471755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.492855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbaf9d-530f-4a4f-b584-ff5b7978d4a5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.526577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.527261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.527541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.527809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.528531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:25.528853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.530306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.539197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd7d31-7c86-49ff-9393-27da43aab2d4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.541724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 251/519] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:25.542510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.578746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.579562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.579846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.580157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.587980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.589472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:25.605762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:25.606658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 269/520] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:25.607934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.645903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f70565-c383-40d9-aacb-fd634816683d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.647503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.682533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.683233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.683515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.683785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.684509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:25.684811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.686343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.697890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 252/521] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:25.698779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.708994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.717199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccfccf-0bbb-47c0-9cf9-7cfb484db193 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.752226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.753088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.753369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.753649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.754435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:25.754766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.756216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.774931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 270/522] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:25.775543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.798992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a21648-c008-4997-be90-a425ee11d29b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.832841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.833503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.833785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.834081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.834975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:25.835290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.836750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.853707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 253/523] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:25.854902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.876101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.880566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9a2c2-d847-42fb-a718-868ccff00148 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:25.915675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:25.916345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:25.916627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:25.916902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:25.917606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:25.919086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:25.938927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 271/524] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:03:25 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:25.939734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:25.953138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:25.966049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3223f4b-5e66-4bcf-8bd1-aef84e4a38ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.012699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.013736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.014020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.014302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.015227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:26.015540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.017059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.037528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 254/525] 10.222.0.64 () {56 vars in 1139 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:26.038236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.068450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1be97b-fd89-4a50-88cf-4b209e1ea020 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.106295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.107172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.107484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.107765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.118142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.119022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.126007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:26.136186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:26.137140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:26.137972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 272/526] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:26.138560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.158292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903af9a9-9bf4-4648-8fac-9b3434eab76c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.193475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.194122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.194400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.194668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.195531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ac88daa9b11b4e13b89f3f981686ac72) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:26.195841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.197292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.206898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 255/527] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/role_assignments?user.id=ac88daa9b11b4e13b89f3f981686ac72&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:26.206898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.218538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.264674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535b3f8-4858-4203-abe2-ebaf1e5c3565 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.318984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.319682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.320364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.320639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.350146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056c81-f81e-4c8b-b62b-0556e8e3146e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.399357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.400412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.401202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.401597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.411441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.413208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.421730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.487091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.495329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.497693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.521515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 273/528] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:26.522300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.616945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 256/529] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:26.616945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.639791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dc75c-0856-4e40-a86f-c3ce68f55da3 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.658548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.678430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.679470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.680023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.680484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.681456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:26.681942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.683698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.704649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 274/530] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:26.705909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.727665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2e60c-49e4-436f-b8c8-f00cb59a4a4e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.743087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.763831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.764811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.765056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.765330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.776190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.777735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:26.803882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:26.804829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:26.805650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:26.805650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:26.805650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:26.805650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:26.809925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 257/531] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:26.811122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.822363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062009-f4bf-4edc-b23c-8c86ac1ec2a7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.859684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.860975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.861459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.861931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.874199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.876189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:26.894989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:26.895879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:26.896723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:26.896723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:26.896723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:26.896723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:26.903338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 275/532] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:26.904509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:26.908688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:26.915564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d6848-dbde-4079-9261-44ff0950677e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:26.952344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:26.953288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:26.953288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:26.953573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:26.954361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=bb0cbf8b93e741c2b500cbf1cb1ad5f6) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:26.954669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:26.956143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:26.966732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 258/533] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/role_assignments?user.id=bb0cbf8b93e741c2b500cbf1cb1ad5f6&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:26.966732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.028717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:27.072820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e220111-4437-4ac4-97c7-c78ea3b070e7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.077771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8971a9-6254-4b97-8efc-fc6cc7b22e09 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.114529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:27.115289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:27.115573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:27.115852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:27.116645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:27.116959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:27.119114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:27.138503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 259/534] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:27.144541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.189420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:27.190154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:27.190434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:27.190724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:27.192768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:27.193079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:27.197337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:27.206808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:27.218859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 276/535] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:27.224386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.297800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67ed06-5966-466a-b34a-1dc972e5b2df None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.333118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:27.333916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:27.334193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:27.334462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:27.346746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:27.346746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:27.372539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:27.373532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:27.374497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:27.376504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 260/536] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:27.386041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.407519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bcc1f-f433-4854-b285-436c2151ad68 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.468015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:27.468827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:27.469103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:27.469374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:27.489132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:27.490668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:27.524552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:27.525570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:27.526403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:27.533748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 277/537] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:27.535344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.541102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:27.568175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:27.602820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1edbc-8db4-4341-acdb-122d7da16530 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.670441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:27.671532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:27.672044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:27.672518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:27.673517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:27.674023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:27.677463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:27.726741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 261/538] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:27.728512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7e6b0-d243-4b57-84ca-651b6080b97e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.740336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.768416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:27.770838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:27.771556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:27.772686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:27.773711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:27.774425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:27.776913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:27.802062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 278/539] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:27.808902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:27.915231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:27.941641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd618bd-a31b-44c2-ab15-e2e2b73ba35b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:27.994197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf7cf1-66e1-41fe-8d30-bd44f339082e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:28.010538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.011268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.011544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.015023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.015880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ad007752722c453bafd9e3a8b30226a8) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:28.016193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:28.017651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc9b4b-e9e3-4440-a97c-f51a0516c7dc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:28.030414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 262/540] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:27 2022] GET /identity/v3/role_assignments?user.id=ad007752722c453bafd9e3a8b30226a8&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:28.052205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.052877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.053155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.053425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.054265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=dfd4560d3f144b339adb62b9ddf118ad) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:28.054572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:28.056064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1e477-9fc7-417e-b007-a91dc4cb10e2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:28.063569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 279/541] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:27 2022] GET /identity/v3/role_assignments?user.id=dfd4560d3f144b339adb62b9ddf118ad&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:28.081214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28788b-0931-428c-be60-3792451ee4f7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.081540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28788b-0931-428c-be60-3792451ee4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.081785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28788b-0931-428c-be60-3792451ee4f7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.083604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 263/542] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:28.089572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2e0ea-25e1-411b-bc12-3f67dc7fb070 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.090112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2e0ea-25e1-411b-bc12-3f67dc7fb070 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.090112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2e0ea-25e1-411b-bc12-3f67dc7fb070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.109254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bac2e0ea-25e1-411b-bc12-3f67dc7fb070 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:28.110559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bac2e0ea-25e1-411b-bc12-3f67dc7fb070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:28.209350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 280/543] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:28.251086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28788b-0931-428c-be60-3792451ee4f7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:28.254287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:28.373514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd4631-52b3-4b54-b3dc-a14214b3e1ab None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.373836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd4631-52b3-4b54-b3dc-a14214b3e1ab None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.374079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd4631-52b3-4b54-b3dc-a14214b3e1ab None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.375806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 281/544] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:28.381159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.381447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.381722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.383340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28788b-0931-428c-be60-3792451ee4f7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:28.403284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:28.404568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:28.429345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.430516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.430986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.431389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.444989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:28.454503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:28.455608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:28.456547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:28.459512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 264/545] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:28.468062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:28.515494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:28.550078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 282/546] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:28.571191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:28.584454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:28.630694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f226d-c3f7-4fcf-8456-105d6bb1379e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:28.693187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab1f8e-bae1-4a0a-89df-c3e339bf4b4f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:28.728623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.729719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.730000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.730273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.741097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:28.747404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.748307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.748793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.748887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.749585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:28.749895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:28.750500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:28.751416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:28.752204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:28.752803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:28.752881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 283/547] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:28.759669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:28.780285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 265/548] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:28.786255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:28.843142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:28.902912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cf2a7-c225-4f80-98e8-4c8be26f9f84 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:28.941525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487ccd-5e79-48aa-845c-57163555d994 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:28.971085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:28.981195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.984535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.985447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.986380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:28.992095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:28.993043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:28.998488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:28.999256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.000232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:29.000760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.002458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.009128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.013666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.024529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 284/549] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:29.025691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:29.040783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:29.042175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:29.043252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:29.043252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:29.043252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:29.043252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:29.054980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 266/550] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:29.058202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.233025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675ed7-915b-436e-a5c6-fc085370de66 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.249586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:29.258873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1724d-cc4d-4544-884f-2544965feec6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.289140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.289954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.290238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.290515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.301666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.303193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.318146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.319198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.319674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.320261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:29.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:29.321829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:29.322864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:29.322864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:29.322864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:29.322864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:29.322864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 285/551] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:29.323773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:29.324276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.325871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a71977-2d22-4243-9e84-2933c28b1fcb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.334726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.358107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 267/552] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:29.360298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85810f-d3c8-4e70-aede-30174513a6b8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.360594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85810f-d3c8-4e70-aede-30174513a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.361044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85810f-d3c8-4e70-aede-30174513a6b8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.362912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 268/553] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:29.363627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85810f-d3c8-4e70-aede-30174513a6b8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.412307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:29.464791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49920-b3cf-490c-b63f-c4c871cba12e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.501487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85810f-d3c8-4e70-aede-30174513a6b8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.503437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.504320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.504604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.504883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.505617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:29.505927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.507401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ad61a-4d5e-4639-af52-4f5eff700619 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.524562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 286/554] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:29.524562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.524562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.524562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.539970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:29.541231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:29.548410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.550000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.550506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.551012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.556907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:29.562551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.565521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:29.584835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:29.585863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:29.586802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:29.590321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 269/555] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:29.591232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.620908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 287/556] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:29.623600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.713423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:29.720224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b01d-dea9-405b-aea5-ca42699140a7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.748540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c7439-de7a-4167-8f91-f99b1aa6680b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.756796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.757833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.757833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.758127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.765720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.767268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:29.782184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:29.783443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:29.784820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:29.787911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 270/557] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:29.788798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.791099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.792124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.792644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.793168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.794243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:29.794944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.796763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.816325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 288/558] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:29.817602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.829282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:29.894845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b79c5e-d6b1-4fe0-b05a-99add3e923c7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.928998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18397c-64b1-4376-8358-9a9aceca0630 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:29.929507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.930364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.930646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.931002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.941967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:29.949742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:29.951197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:29.952038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:29.956361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 271/559] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:29.957315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:29.971488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:29.972520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:29.973033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:29.973564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:29.974773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:29.975286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:29.976939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:29.977450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:29.998596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 289/560] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:29.999806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.070387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46b3c-5722-4144-9668-533782b2976d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.071301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.105934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.106587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.106901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.107212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.110827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c8eab-a228-4c97-a58f-7206d99138bc None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.148164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.149246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.149754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.150278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.151896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:30.152434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.154161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.171069 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 290/561] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:30.172515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.178909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.180405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.198613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.283358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9489-7d32-497c-8087-3c13660cea62 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.286999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 272/562] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:30.287964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.314441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.324669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.325706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.326044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.326324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.338398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.339951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:30.357447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:30.358401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:30.364344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:30.364344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:30.364344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:30.364344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:30.364344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 291/563] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:30.364344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.394109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80738c-d91f-4947-92e3-d59ed0776f88 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.412436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.434126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.435287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.435596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.435870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.446174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.447705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b4e559c2-7e6c-4cd5-be03-a8098c37173a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:30.465961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:30.467193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:30.468624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:30.468624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:30.468624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:30.468624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:30.468846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676349fb-45b1-462c-8619-d4409a50cfc2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.468931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 273/564] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:30.469001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6802b-93db-469d-809a-85e3cd20fdd0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.469283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6802b-93db-469d-809a-85e3cd20fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.469552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6802b-93db-469d-809a-85e3cd20fdd0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.475019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 274/565] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:30.475824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6802b-93db-469d-809a-85e3cd20fdd0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.509341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.510497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.511012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.511471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.524940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.526659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:30.545026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:30.546228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:30.547235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:30.547235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:30.547235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:30.547235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:30.552768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 292/566] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:30.552768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.584023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6802b-93db-469d-809a-85e3cd20fdd0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.584981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.622322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.623181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.623366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.623621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.624345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:30.624649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.626378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc792a92-18fa-4393-b0fc-234d2744dde2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.638049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 275/567] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:30.639225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee62a0-ce0d-48db-b2bc-8d3a3390e799 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.639514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee62a0-ce0d-48db-b2bc-8d3a3390e799 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.639790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee62a0-ce0d-48db-b2bc-8d3a3390e799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.657373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dfee62a0-ce0d-48db-b2bc-8d3a3390e799 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:30.658571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfee62a0-ce0d-48db-b2bc-8d3a3390e799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:30.660381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a77b8-b068-4ae8-93e6-aebaf77ac363 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.682174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.694643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.695363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.695672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.695943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.696666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:30.696971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.698361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.712068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 293/568] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:30.712898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.730471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 276/569] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:30.731892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57df0b-8368-4bac-b7c8-7113b3071314 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.732251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57df0b-8368-4bac-b7c8-7113b3071314 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.732531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57df0b-8368-4bac-b7c8-7113b3071314 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.737798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 277/570] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:30.737798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57df0b-8368-4bac-b7c8-7113b3071314 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.808871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.818071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e821e-1430-4242-95a6-e0bafd8d659d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.837079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57df0b-8368-4bac-b7c8-7113b3071314 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.850913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.851581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.851857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.852134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.852852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:30.853157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.854919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.865937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 294/571] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:30.866730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:30.875935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:30.876731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.877008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.877279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.884334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:30.885783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-43658582-03a4-45dc-a50d-8bf9b1bbf034 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:30.899839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:30.900897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:30.902084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 278/572] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:30.902813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:30.902813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:30.902813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:30.919248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:30.921029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:30.934050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:30.972321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87ec39-4795-41d3-a33b-658337ba688a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:30.988830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 279/573] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:30.989643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.006348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.007187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.007472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.007748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.015464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.016993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:31.030892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:31.031843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 295/574] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:31.037585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.048910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.102370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce551567-b94d-4c7f-b784-faae4fcc34ad None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.132550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce0dd0-e4d6-498e-881d-a529b1331ecd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.137059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.137866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.138147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.138531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.148793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:31.156613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:31.157554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:31.158377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 280/575] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:31.159112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.171100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.171905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.172187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.172461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.173413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.179795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.181267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:31.195459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:31.196366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:31.197232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 296/576] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:31.197714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.266185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7494d78-0d36-4651-9ed4-917a87ea1af3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.268088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.297134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587e3dd-8255-4ccb-a087-ec3a1662c09a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.299846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.300502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.300784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.301067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.301860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:31.302171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.303655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.316025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 281/577] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:31.316597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.333212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.334018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.334302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.334575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.345267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:31.352804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:31.353768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:31.355042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 297/578] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:31.355731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.384224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.418063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b329-5f0d-4d46-9ac0-18c151df64a7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.452584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.453253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.453535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.453820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.454652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:31.454991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.456452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.457006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7973a-1145-4f62-835a-a79a117ee367 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.475379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 282/579] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:31.476200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.487981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.493336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.494009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.494286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.494575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.495350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:31.495653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.497015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.505254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 298/580] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:31.505993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.576224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d9d78-1975-480d-a97b-35fe014d4d41 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.601711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.611352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.611679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fcd73-413c-43bf-8a91-5600007e73aa None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.611950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.612229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.612502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.613285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:31.613595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.615073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.626433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 283/581] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:31.627378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.653446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.654439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.654827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.655226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ac88daa9b11b4e13b89f3f981686ac72/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.669442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=ac88daa9b11b4e13b89f3f981686ac72, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.671408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.683896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 299/582] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:31 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ac88daa9b11b4e13b89f3f981686ac72/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:31.684694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.743422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a4461-8f50-44d7-a58c-e5aa14c97dfe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.751677 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.785202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.785892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.786174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.786992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.787765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:31.788123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.789905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.803986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 284/583] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:31.803986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.819428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2331c-2a6b-4973-acce-94e86eb08c08 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.862660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.863428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.863713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.863986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.866110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:31.875987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.877912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:31.897010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:31.897972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:31.898874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:31.898874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:31.898874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:31.898874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:31.898874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 300/584] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:31.899164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:31.924440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72b1b9-0f74-46da-8e85-8e4145d46305 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:31.959963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:31.960828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:31.961085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:31.961365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:31.973045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:31.974634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:31.993064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:31.994003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:31.994951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:31.994951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:31.994951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:31.994951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:31.994951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 285/585] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:31.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.002743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.013658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1a2e4-ec75-4a2c-a7e6-2e6a36fe0234 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.049779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.050450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.050874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.051155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.052029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=19fd34c5b9e34440b7c3752ff1cdf4fe) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:32.052345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.053782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.066735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 301/586] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/role_assignments?user.id=19fd34c5b9e34440b7c3752ff1cdf4fe&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:32.066735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.114888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.120873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72dea-9b8b-4cf7-acf5-04560f5d8c2c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.161785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.164106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.164106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.164366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.181056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.183532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.204232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9de8a-5ba7-406a-90a0-567f42458e76 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:32.210445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:32.211448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:32.212329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:32.212329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:32.212329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:32.212329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:32.215196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 286/587] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:32.217508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.253286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.254460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.255415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.255528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.257607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:32.258022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.260123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.273591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 302/588] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:32.274353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.345874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92ac0-4724-4fea-9937-bb3e9f9c61e9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.375126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.387776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.389767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.390256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.390745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.391682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:32.392203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.393785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.407499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 287/589] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:32.408724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.410054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcfc06-f371-44c8-a572-50d43631a510 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.425599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.446754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.447469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.447740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.448017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.448755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:32.449067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.451423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.468528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 303/590] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:32.468528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.541545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697d22-9f61-4e3b-a7e7-b25d74870438 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.574302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.589690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.590481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.590780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.591050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.598917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.600375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.601032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b59a7-dd04-40dc-b31e-f7f2164339c4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:32.615315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:32.616252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:32.617139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:32.621371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 288/591] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:32.622222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.641329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.642459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.643007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.643484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.651428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.653223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:32.668520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:32.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:03:32.670959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:32.673679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 304/592] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:32.674809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.700487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.734366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f971ee-e016-434f-a3ac-a295baa652e7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.769720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.770556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.770862 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.771131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.779174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.780911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.787752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c4e32-e2a6-46c0-b9de-b7a058c52353 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:32.796965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:32.797937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:03:32.798887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:32.803081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 289/593] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:32.804077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.836967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.838080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.838561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.839074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.840149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:32.840655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.842501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.852843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.858855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 305/594] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:32.860033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:32.935758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5004328-06b7-42a4-8d5a-a86286faa624 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.966371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:32.972654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:32.973598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:32.974078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:32.974543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:32.975771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:32.976302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:32.978235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:32.986000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c7633-a252-47d1-a3c5-ddfe9acd9f48 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:32.998879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 290/595] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:33.000006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.026522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.027238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.027521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.027800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.028808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:33.029179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.030981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.051757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 306/596] 10.222.0.64 () {56 vars in 1139 bytes} [Fri Jun 24 10:03:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:33.052986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.116772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:33.130770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf763f-fc08-4069-87a1-3c98be693915 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.169144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b07e0-7654-4d0c-99d1-fcef1f27c14d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.169344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.170661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.171168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.171713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/bb0cbf8b93e741c2b500cbf1cb1ad5f6/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.188090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=bb0cbf8b93e741c2b500cbf1cb1ad5f6, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.189981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.223137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.224345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.224947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.225422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.226877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 291/597] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:32 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/bb0cbf8b93e741c2b500cbf1cb1ad5f6/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:33.228076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.239837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.242969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:33.266775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:33.267729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:33.268614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:33.268614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:33.268614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:33.268614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:33.270923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 307/598] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:33.277145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.355073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:33.400938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55ce76-b15e-42ab-84e6-be3c352d0fcc None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.414618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:33.465430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.466378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.466910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.467334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.468336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:33.468793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.470323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.472127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a35c7-5d68-48d9-82e1-a3553a955fc9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.487139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 308/599] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:33.492754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.515685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.517236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.517236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.517600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.529698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.531473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:33.553974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:33.555170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:33.556254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:33.556254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:33.556254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:33.556254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:33.556254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 292/600] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:33.561058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.603829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3ee66-fa61-4410-ae8f-bd0d05bbcc5c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.640396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.641512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.641944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.642393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.650031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.651941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.663096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:33.667848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:33.668963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:33.670143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:33.670685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a28ea-ebd2-4eb7-81a5-281a7286b2dd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.672397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 309/601] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:33.678036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.706727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.707508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.707864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.708253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.709060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:33.709909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.711151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.719389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:33.728142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 293/602] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:33.735494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.853136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab2102-79f7-4a59-abf8-8c3ce56fd5b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.879232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb79a4-bb0c-492f-83ba-43fdaa56c238 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:33.892793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.893699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.894038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.894421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.902529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.904229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:33.920965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:33.922085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:33.923207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:33.924843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 294/603] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:33.930320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:33.948593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:33.961427 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:33.962442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.963002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.963464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.964481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:33.965059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:33.967028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1ed81-dfaf-4d8e-8ffd-fdd9ff78daf2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:33.982876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 310/604] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:33.986546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac015-1852-4ba3-8b5c-1a087bee91a6 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:33.986937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac015-1852-4ba3-8b5c-1a087bee91a6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:33.987244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac015-1852-4ba3-8b5c-1a087bee91a6 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:33.992860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 311/605] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:33.993725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac015-1852-4ba3-8b5c-1a087bee91a6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.058134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e366fe-e0ad-463c-a1ea-a87f0518c694 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.095651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.096187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.096535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.096881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.097753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:34.098135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:34.099943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cefe9-c5eb-4ef6-adb3-0200697a98bf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:34.102680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:34.118238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 295/606] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:34.118238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f2a0-2ad9-4684-b8cf-f822e4d1aac8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.118238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f2a0-2ad9-4684-b8cf-f822e4d1aac8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.118238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f2a0-2ad9-4684-b8cf-f822e4d1aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.120673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac015-1852-4ba3-8b5c-1a087bee91a6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.134091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2f37f2a0-2ad9-4684-b8cf-f822e4d1aac8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:34.135445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2f37f2a0-2ad9-4684-b8cf-f822e4d1aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:34.159301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.160564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.161028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.161515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/dfd4560d3f144b339adb62b9ddf118ad/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.178745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=dfd4560d3f144b339adb62b9ddf118ad, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:34.180774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:34.198654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 312/607] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:33 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/dfd4560d3f144b339adb62b9ddf118ad/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:34.200056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.221945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 296/608] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:34.230634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c33753-6c10-429e-8776-04a3b9a3f2a4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.230946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c33753-6c10-429e-8776-04a3b9a3f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.231229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c33753-6c10-429e-8776-04a3b9a3f2a4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.232709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 297/609] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:34.244355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.244542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.244867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.252907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:34.273285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:34.274550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:34.340563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966ea89-1391-4e1b-87fa-b7301d320cdd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.351126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 298/610] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:34.352164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.391642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.393030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.393812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.395043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ad007752722c453bafd9e3a8b30226a8/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.414500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:34.421322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=ad007752722c453bafd9e3a8b30226a8, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:34.424052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:34.439401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 313/611] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:34 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ad007752722c453bafd9e3a8b30226a8/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:34.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.478814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2b78f-6217-4436-9965-e9bbb158a08e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.513480 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:34.528056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.528911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.529260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.529539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.554823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:34.565469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:34.566466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:34.567358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 299/612] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:34.573071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.610590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2724f6c-6eff-4d45-b489-c832e4ca3250 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.651168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.652398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.652886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.653364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.667599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:34.676492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:34.677488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:34.678319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:34.681323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 314/613] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:34.687715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.738298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bc45a-7311-4de1-a303-0756c9ef5108 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.757051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:34.785003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.785594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.785870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.786193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.786982 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:34.787325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:34.788701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:34.798832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 300/614] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:34.804113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.818957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31019c-238d-4b15-841f-893ac4abd6ea None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:34.854937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:34.856176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:34.856701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:34.857205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:34.858338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:34.859050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:34.860763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:34.870153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 315/615] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:34.875578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:34.917930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:34.979621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c769b-a084-4281-b573-7c08d9e18e66 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.033830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c447f-afcc-4064-a1fb-bbdc2b18fa0e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.052831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.053562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.053843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.054115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.059142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:35.075356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.076833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.077418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.077904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.081258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.083081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.101880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.104946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:35.114508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:35.115448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:35.116381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:35.116381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:35.116381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:35.116381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:35.119162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 301/616] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:35.124345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:35.138314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:35.139397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:35.140272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:35.140272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:35.140272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:35.140272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:35.142714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 316/617] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:35.151705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.284491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01d959-4994-4cf7-a8db-5a3e4bd0611c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.292677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:35.313595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a963a-404a-4665-998c-b22554a3fef0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.339403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.341268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.342923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.343407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.345167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:35.347128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.348744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.372294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 302/618] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:35.377834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.384382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.385046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.385331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.385597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.386326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:35.386628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.388030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.409398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 317/619] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:35.416692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.463076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:35.577967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090cde-fdba-4ecd-9c92-f333477216a6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.585022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7ee1d-1c3b-4df5-b25c-c898573d3909 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.621597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.622882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.623368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.623838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.631960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.633836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:35.649509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:35.650627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:35.651640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:35.654589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 318/620] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:35.659911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.661762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.663619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.665739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.666223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.681451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.683844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7bc848a7-69d4-45c5-9cfe-1ce762c8d835 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:35.701955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:35.702967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:35.703921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:35.707045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 303/621] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:35.708543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef17a-7f28-4a1a-b5ca-adb097dc8fa7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.709046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef17a-7f28-4a1a-b5ca-adb097dc8fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.709529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef17a-7f28-4a1a-b5ca-adb097dc8fa7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.711312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 304/622] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:35.717297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef17a-7f28-4a1a-b5ca-adb097dc8fa7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.748194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:35.775190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:35.792490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba90f9-9fc5-4b55-915d-5884c76dfe6c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.829388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.829988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.830269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.830541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.831425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:35.831734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.833280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ed04d-6ca9-41e7-a16a-377a4291f9be None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.834000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef17a-7f28-4a1a-b5ca-adb097dc8fa7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.856387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 319/623] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:35.856387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.856387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.856387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.873246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:35.874543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:35.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.879422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.879964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.880440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.881617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:35.882204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.884170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.908543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 305/624] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:35.910173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.948955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 320/625] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:35.954222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.022501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.030587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.049564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7ff78-afcc-4ee5-b0a8-274a2df1ac25 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.056645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73be489-3900-4ed2-897b-bbfa965d01dc None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.087435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.088659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.089087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.089550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.093667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.094330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.094621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.094928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.101698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.103538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:36.123440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:36.124578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:36.125622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:36.125622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:36.125622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:36.125622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:36.132227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 306/626] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:36.133283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.166124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.167790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.241834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.243334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478dd35-ff91-49c1-9e6b-6a63997a22ff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.278797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 321/627] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:36.280430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.284334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.285336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.285767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.286186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.289331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.299782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:36.308005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:36.309162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:36.310191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:36.314612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 307/628] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:36.316415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.402564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9122a9-ab3d-4434-b604-c5fa9a195d1f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.430965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7752975-23b2-4e3f-9830-7c5227860369 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.440104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.441373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.441894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.442335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.443338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:36.443801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.445333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.462589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 322/629] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:36.463820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.468709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.469521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.469795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.470062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.482413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.484006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.499989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:36.504612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:36.512324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:36.513275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:36.513275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:36.513275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:36.513275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:36.513275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 308/630] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:36.513275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.559908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.583492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b8e27-733c-477e-89ae-ec6cdad41cbb None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.614146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1e5a-e6b6-4953-9075-96290ab9c539 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.619357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.620032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.620309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.620581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.621460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:36.621649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.623059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.632213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 323/631] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:36.632805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.655038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.655855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.656133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.656412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.664048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.665543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4ce04c0b-88f9-49a7-9907-651478380131 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:36.682244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:36.683536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:36.684777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:36.689534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 309/632] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:36.691251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ff65-7ded-47d2-8807-d4697ed6535f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.691540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ff65-7ded-47d2-8807-d4697ed6535f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.691820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ff65-7ded-47d2-8807-d4697ed6535f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.697426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 310/633] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:36.698258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ff65-7ded-47d2-8807-d4697ed6535f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.740547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.747903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0417ca-95ca-4cb8-8588-44c1b08821ae None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.784388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.785066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.785383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.785662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.786393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:36.786748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.788110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.805385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 324/634] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:36.806306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.812523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ff65-7ded-47d2-8807-d4697ed6535f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.824218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:36.852408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.853196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.853479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.853749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.864843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.866322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-594cc656-6eba-4a31-abdd-4d152f4a0f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:36.886455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:36.887699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:36.889190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:36.889190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:36.889190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:36.889190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:36.889190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 311/635] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:36.889942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee96b-87b2-48ca-bc16-022b22d44f1b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.890223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee96b-87b2-48ca-bc16-022b22d44f1b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.890495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee96b-87b2-48ca-bc16-022b22d44f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.909287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e75ee96b-87b2-48ca-bc16-022b22d44f1b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:36.910403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e75ee96b-87b2-48ca-bc16-022b22d44f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:36.923917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059e0a6-859f-4a3d-ac24-599a51c05e73 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:36.962673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:36.963365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.963648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.963925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.965432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:36.965945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:36.968191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:36.981262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 325/636] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:36.982260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:36.985247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 312/637] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:36.987191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc67f5c-73b7-48e7-b39c-a818d5bc1bee None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:36.987558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc67f5c-73b7-48e7-b39c-a818d5bc1bee None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:36.987836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc67f5c-73b7-48e7-b39c-a818d5bc1bee None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:36.989197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 313/638] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:36.990199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc67f5c-73b7-48e7-b39c-a818d5bc1bee None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.009485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.100799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.103882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14683d64-0279-453a-b489-8087f27071c1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.112637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc67f5c-73b7-48e7-b39c-a818d5bc1bee None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.143290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.144096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.144371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.144643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.148613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.149269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.149549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.149821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.150552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:37.150890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.152298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.152489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.153940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.165364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 314/639] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:37.166093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-6bb84fcd-474e-4bc2-a212-48ba74a40bf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:37.169040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:37.170180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:37.171238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:37.174908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 326/640] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:37.176399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.176682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.176984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.195209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:37.196749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:37.269106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 327/641] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:37.270100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.273832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.276615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0ede1-2326-4c18-933e-0045b1d959bf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.326241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.327171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.327453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.327728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/19fd34c5b9e34440b7c3752ff1cdf4fe/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.343075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=19fd34c5b9e34440b7c3752ff1cdf4fe, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.344544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.353606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.359246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 315/642] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:37 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/19fd34c5b9e34440b7c3752ff1cdf4fe/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:37.360206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.390790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612cf48-7c71-4fb4-98a8-1b9cfb1bf02f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.448137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.449547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.449849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.450126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.465659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:37.477204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:37.478321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:37.479988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:37.484155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 328/643] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:37.485237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.501248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a8d0-15ed-4ccf-aa18-98d5b6230260 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.549264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.550078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.550358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.550629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.558608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.560123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.570137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:37.575704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:37.576962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 316/644] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:37.582999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.627237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2ae2b-e7b3-4b59-8594-d9a2f730a996 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.648039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.675547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.676651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.677629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.677922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.679081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:37.679404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.681031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.697610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 329/645] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:37.698928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.704167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd05b32-22c5-4ae4-93e1-69a180821d14 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.744172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.744991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.745260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.745530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.756691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:37.764416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:37.765347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:37.766180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 317/646] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:37.766888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.814204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf1b45-3d19-41f3-a72e-7646d4d97b2f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.840335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.856123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.856814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.857130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.857413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.858167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:37.858478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.860456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.874930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 330/647] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:37.874930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.888955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024f8ea-9873-4c16-8715-0f3167e69dba None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:37.908912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:37.923750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:37.924426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:37.924701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:37.924977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:37.925834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:37.926132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:37.927625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:37.945041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 318/648] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:37.945755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:37.990663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f8592-eeec-4004-adf7-3acec1ebc767 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.024216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.024883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.025164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.025440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.026264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ac88daa9b11b4e13b89f3f981686ac72) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:38.026576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.028064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.042737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 331/649] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/role_assignments?user.id=ac88daa9b11b4e13b89f3f981686ac72&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:38.042737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.063475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab902239-9111-4ea1-8031-e081c9250c93 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.098281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.103236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.104217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.105001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.105448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.106352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:38.106830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.109014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.123070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 319/650] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:38.124363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.163608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30b93-410b-438c-ab44-6ef4e825b931 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.185768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.205571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.206385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.206666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.206971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.219516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.220987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:38.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:38.240991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:38.242085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:38.242085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:38.242085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:38.242085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:38.242085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 332/651] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:38.242085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.262486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbde6c-f030-4b60-968a-5fae6e2a6c7a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.307557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.309105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.309398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.309669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.368255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc35a-7cd6-4ab1-ade3-5727c2c7d618 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.370262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.402373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.403869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.405912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.406977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.407419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.407857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.419475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.421436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:38.439626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:38.440713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:38.441763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:38.441763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:38.441763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:38.441763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:38.446827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 333/652] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:38.447849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.450110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.531695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 320/653] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:38.532963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.573399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c32c-da64-42e6-b63b-044168e37fd4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.611454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.612137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.612413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.612713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.614020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:38.614345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.615937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.636319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 334/654] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:38.636319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.651119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.667960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f41db9-2461-497c-9bbd-059850ec9ce8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.721934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.723478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.724211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.724667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.725840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:38.727108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.728504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.739453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.745842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 321/655] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:38.747277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.762536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81461c7c-cd3f-4efa-b219-2a2219f9e243 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.801896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.802733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.803093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.803434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.814960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.816789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:38.837356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:38.838292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:38.839234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:38.839234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:38.839234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:38.839234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:38.839234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 335/656] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:38.840101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:38.878385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34b892-d08e-4b6c-a3b2-cc64b5283030 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.917978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:38.933222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:38.934685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:38.935437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:38.935899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:38.945320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:38.947505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:38.962655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf685e8-5bd1-4594-8119-a34d54487c0a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:38.968035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:38.968990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:03:38.969924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:38.972942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 322/657] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:38.973940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.005353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.006193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.006481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.006776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.015487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.017019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:39.033655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:39.034661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:39.035632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 336/658] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:39.036242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.050680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.096780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1126d97-edbd-4667-a874-c3524e3fb448 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.143296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.144341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.144861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.145355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.146749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:39.147539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.149628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.159343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccc4d-f1ab-489e-aaa5-5ffb38f7da37 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.165757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 323/659] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:39.167370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.188760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.199500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.200424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.200906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.202000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.203148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:39.203640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.205466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.226907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 337/660] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:39.227104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.284988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41ec5a-6523-4bcf-a030-3f3108a8aa5a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.308389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.326091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.327807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.328293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.328741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.329751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:39.330326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.332091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.351815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 324/661] 10.222.0.64 () {56 vars in 1139 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:39.353062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.366993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6b2c2-3046-4b4e-82d4-f88b9f18200b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.410895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.411708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.411994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.412269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.421534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.423494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:39.440197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:39.441232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:39.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 338/662] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:39.442596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.472580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bba52-fd7c-4f60-8d21-da573de0643e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.497387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.512092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.512754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.513051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.513296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.555648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f0fd7-e1b4-4707-8b32-adc9219ec963 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.563079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.587536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.589441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.594206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a31a0-e662-40e3-bf55-ba88b11056eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.594922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a31a0-e662-40e3-bf55-ba88b11056eb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.595212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a31a0-e662-40e3-bf55-ba88b11056eb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.595493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a31a0-e662-40e3-bf55-ba88b11056eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.666239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a31a0-e662-40e3-bf55-ba88b11056eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.667735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a31a0-e662-40e3-bf55-ba88b11056eb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.703888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 325/663] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:39.704767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.718653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.777188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 339/664] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:39.778850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d3cab-b773-4ddf-a3a9-be273705b9c4 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.779492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d3cab-b773-4ddf-a3a9-be273705b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.779955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d3cab-b773-4ddf-a3a9-be273705b9c4 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.781735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 340/665] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:39.782684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d3cab-b773-4ddf-a3a9-be273705b9c4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.790937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.814090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa21680-1c50-4cc0-939b-c6b0f3a0acfd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.849903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.850929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.851364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.851639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.852436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:39.852743 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.854193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.871548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 326/666] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:39.872859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:39.892498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d3cab-b773-4ddf-a3a9-be273705b9c4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.927398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:39.928542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.928938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.929217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.939261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:39.940714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:39.943326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-536e03cb-7acd-407a-bca6-473efea6495c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:39.959061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:39.960244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:39.961344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:39.961344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:39.961344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:39.961344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:39.966686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 341/667] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:39.967999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:39.968286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:39.968559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:39.982621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b1c1-75ca-4d99-889d-50acb2c1e4ed None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:39.985590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:39.986843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:40.019308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.020111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.020401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.020711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.027477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.032722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.034198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:40.052067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:40.053003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:40.053908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:40.053908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:40.053908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:40.053908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:40.053908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 327/668] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:40.054178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.074869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 342/669] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:40.074869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.174426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6416e2-272c-459b-9abb-e93c8753e6fb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.175645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daba87-380a-4f58-b236-b3b58566bf61 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.203726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.211736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.212404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.212513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cf981-7db3-4133-b640-fc003803774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.212681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.212953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.213111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cf981-7db3-4133-b640-fc003803774c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.213394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cf981-7db3-4133-b640-fc003803774c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.213669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cf981-7db3-4133-b640-fc003803774c None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.213784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=bb0cbf8b93e741c2b500cbf1cb1ad5f6) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:40.214092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.214399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cf981-7db3-4133-b640-fc003803774c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:40.214727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cf981-7db3-4133-b640-fc003803774c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.215574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.216119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cf981-7db3-4133-b640-fc003803774c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.221730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 328/670] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/role_assignments?user.id=bb0cbf8b93e741c2b500cbf1cb1ad5f6&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:40.223608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.232133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 343/671] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:40.232133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cf981-7db3-4133-b640-fc003803774c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.292317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.342276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda214d-1524-4bb3-a0c0-ff0e0cd55037 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.358858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cf981-7db3-4133-b640-fc003803774c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.385497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.386177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.386464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.386792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.388454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:40.388778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.390559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80adfcfe-02e9-4809-b45d-82069eb76d09 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.404236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 329/672] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:40.405348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4810b-75b5-4a5b-b57f-b86d4443b61f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.405623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.405748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4810b-75b5-4a5b-b57f-b86d4443b61f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.405913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4810b-75b5-4a5b-b57f-b86d4443b61f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.406387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.406675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.406979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.411462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 330/673] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:40.411462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4810b-75b5-4a5b-b57f-b86d4443b61f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.422462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:40.430196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:40.431232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:40.432065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 344/674] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:40.432771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.485210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.523047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4810b-75b5-4a5b-b57f-b86d4443b61f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.539389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.544751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce5d78-8ab6-46a3-8b0f-15f0237815ec None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.561059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.561862 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.562144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.562423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.569396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.570892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.582062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.583238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.583771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.584233 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-49116383-2ceb-419a-9b6c-10d9f1eff640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:40.584773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:40.585620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:40.586529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 331/675] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:40.587360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.587453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.587737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.597476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.599583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.606296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:40.607590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:40.615717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:40.616814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:40.617927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:40.621676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 345/676] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:40.622544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.684783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 332/677] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:40.687863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.742359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dca69-2888-4789-846d-621bbbeb869b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.750580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.783154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.783980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.784274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.784828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.785868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:40.786178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.788039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.790160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9645d-0a9c-47aa-bc4b-e600933c4bd9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.792536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.798369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 346/678] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:40.799237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.834001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.834669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.834980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.835254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.836588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:40.836588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.837789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.854740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 333/679] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:40.854740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.909517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cdfe1-f9d0-48d1-a62e-af99bb267f9c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.945238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.945981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.946272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.946548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:40.947373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:40.947689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:40.949131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:40.955510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a098-d46c-49ee-b85e-fbe81a7d8d39 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:40.966177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 347/680] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:40.966177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:40.994586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:40.994830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:40.995760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:40.996069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:40.996561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.007277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:41.014950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:41.015893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:41.016738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 334/681] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:41.017418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.042333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.076808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36ab2-de9b-43d3-8da0-8bd330f14dc4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.113185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.114002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.114271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.114548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.117535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49c7c8-0fbe-4d3f-8823-619afc0ca2a0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.124537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.126007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:41.146567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:41.147572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:41.148438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:41.148438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:41.148438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:41.148438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:41.148438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 348/682] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:41.148724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.157264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.157954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.158228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.158505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.159403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=dfd4560d3f144b339adb62b9ddf118ad) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:41.159696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.161145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.167252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 335/683] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/role_assignments?user.id=dfd4560d3f144b339adb62b9ddf118ad&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:41.168023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.252424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.267512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.278454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3f821-ba10-4420-8ec4-c09a9857590d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.282004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9be6f-641d-4e47-a5f2-7bc95dd57a55 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.319441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.320133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.320417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.320703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.321456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:41.321763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.323223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.324060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123d69f-768b-478a-a002-993172f3860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.325086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123d69f-768b-478a-a002-993172f3860a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.325678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123d69f-768b-478a-a002-993172f3860a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.326235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123d69f-768b-478a-a002-993172f3860a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.327351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123d69f-768b-478a-a002-993172f3860a None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ad007752722c453bafd9e3a8b30226a8) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:41.327875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123d69f-768b-478a-a002-993172f3860a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.330303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123d69f-768b-478a-a002-993172f3860a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.342939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 336/685] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:41.342939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.343252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 349/685] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/role_assignments?user.id=ad007752722c453bafd9e3a8b30226a8&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:41.343912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123d69f-768b-478a-a002-993172f3860a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.472868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f380f0c-679c-46d4-a7ca-6d2b9de9d6aa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.488944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123d69f-768b-478a-a002-993172f3860a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.526253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.526631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.527486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.527772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.527877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.528045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.528469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.528748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.529638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:41.530234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.531818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.539922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.541516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.544283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.552570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 337/686] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:41.558321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:41.563789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:41.564770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:41.565765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:41.565765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:41.565765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:41.565765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:41.565765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 350/687] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:41.569889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.575858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.674304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a1de-3e74-4d77-bda9-b1f840bd4744 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.681859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5791c8-4e15-4217-a0a3-38976959cf7e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.714691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.715776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.716205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.716698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.725217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.726892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.729676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.730608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.730763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.731044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.731781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:41.732171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.733615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:41.744330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:41.745479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:41.746585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:41.747155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 351/688] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:41.748971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 338/689] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:41.751995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.754169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.811706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.829738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:41.870382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a8fdf-5775-44b9-840e-990f97acfbeb None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.870925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ce110-2990-4a96-957f-cf65fdc72347 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:41.909975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.911219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.911702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.912175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.920480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.921671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:41.922361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:41.922643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:41.922945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:41.923255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.923813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:41.924289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:41.926035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:41.946131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 339/690] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:41.952027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:41.952249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:41.953148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:41.954162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:41.956760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 352/691] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:41.963284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:42.154098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd52a30-951c-4df8-bd34-7ad711750ff0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:42.160843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38da6ec-908d-47de-87a5-3ecca6dda736 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:42.194420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:42.209686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:42.222070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dc5cd-e4a8-4318-a5a4-ed881e808e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:42.222747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc5cd-e4a8-4318-a5a4-ed881e808e08 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.223033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc5cd-e4a8-4318-a5a4-ed881e808e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.223307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc5cd-e4a8-4318-a5a4-ed881e808e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:42.230679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:42.232469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.232758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.233031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:42.233896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:42.234209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:42.239059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:42.282933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 353/692] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:42.300600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:42.365805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dc5cd-e4a8-4318-a5a4-ed881e808e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:42.367355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dc5cd-e4a8-4318-a5a4-ed881e808e08 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:42.429681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74378c8d-60b6-42ef-8513-2af98365c259 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:42.475500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a348bd-b976-4c3a-ae37-d73e99b057f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:42.476188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a348bd-b976-4c3a-ae37-d73e99b057f2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.476474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a348bd-b976-4c3a-ae37-d73e99b057f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.476749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a348bd-b976-4c3a-ae37-d73e99b057f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:42.567945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:42.579309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 340/693] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:42.581259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75860199-ba14-4e28-8a70-803ea745a5d4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.581549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75860199-ba14-4e28-8a70-803ea745a5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.581827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75860199-ba14-4e28-8a70-803ea745a5d4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:42.587492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 341/694] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:42.587492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75860199-ba14-4e28-8a70-803ea745a5d4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:42.602547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a348bd-b976-4c3a-ae37-d73e99b057f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:42.604844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a348bd-b976-4c3a-ae37-d73e99b057f2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:42.617894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:42.721530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 354/695] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:42.723478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.723965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.724455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:42.732629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75860199-ba14-4e28-8a70-803ea745a5d4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:42.743094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:42.744647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:42.772951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:42.773749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.774038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.774313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:42.787210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:42.788685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:42.815259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:42.816206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:42.817723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:42.817723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:42.817723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:42.817723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:42.817723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 342/696] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:42.817723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:42.826833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 355/697] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:42.828103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:42.836687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:42.926259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:42.952681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50abdf-0349-4058-b18e-a516a34dc4e2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:42.958841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aee09f-8bfe-4977-81d2-21b5db02286b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:42.993618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:42.994890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:42.995343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:42.995875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.001530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.002212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.002495 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.002806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.003549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:43.003858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.005804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.009405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.011090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.020687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 356/698] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:43.021546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:43.030902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:43.031803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:43.032723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:43.032723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:43.032723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:43.032723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:43.035047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 343/699] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:43.036176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.105201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.148306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4070ca-27e4-4592-a672-c425991d29e1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.155836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a83cd2-f3b2-450f-be40-54c03f93d550 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.183593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.184097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.184375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.184649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.195791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.196212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.197826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.203311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.204354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.204814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.205285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.213965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.215759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:43.217499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:43.218384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:43.219321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:43.219321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:43.219321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:43.219321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:43.223758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 357/700] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:43.224717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:43.236547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:43.237563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:43.238445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:43.241593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 344/701] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:43.246885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.339370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dbf23-dc2e-4e0a-a05b-525bb7de6454 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.358788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.382985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.384812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.385325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.385803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.386318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd333-7868-4146-873b-4dcb6e8a15f3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.387024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:43.387537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.390369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.406577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 358/702] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:43.407830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.423082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.423799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.424074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.424356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.425196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:43.425739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.427017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a612a07-00b9-4faf-8e5c-bec77f1b4703 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.443853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 345/703] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:43.445119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0288d-9e2e-457c-8e5e-11922db90120 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.445405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0288d-9e2e-457c-8e5e-11922db90120 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.445677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0288d-9e2e-457c-8e5e-11922db90120 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.451121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 346/704] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:43.451121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0288d-9e2e-457c-8e5e-11922db90120 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.505906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.513409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a161fc6-4ed0-4288-b98e-64c21ba15c2a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.549481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.550161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.550446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.550744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.551547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:43.551858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.553298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.556039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0288d-9e2e-457c-8e5e-11922db90120 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.567330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 359/705] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:43.568139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.597489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.598328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.598568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.598873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.606598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.608239 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.610445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b886fba4-e7e6-483e-95e3-d49058ff7cc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:43.622593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:43.623589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:43.624493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 347/706] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:43.625660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.626015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.626359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.644051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:43.645260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:43.685070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1805ca8-f68c-4ba1-9419-7bfbe7b3cf47 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.725418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.726168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.726368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.726639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.727391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.727820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 348/707] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:43.728553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.728882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.754205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 360/708] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:03:43 2022] POST /identity/v3/users => generated 254 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:43.755826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.793150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.859396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:43.859779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aaaef-be73-4758-a9be-894d167ae1df None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.871358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333405-bbd3-45dc-8c0c-164d04a8eb47 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:43.899669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.900336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.900613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.900882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.901721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=19fd34c5b9e34440b7c3752ff1cdf4fe) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:43.902023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.903516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.907349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:43.908032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.908309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.908582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.909378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:43.909684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:43.911169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f224ba-6d19-4146-8558-00137a91f710 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:43.918983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 349/709] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/role_assignments?user.id=19fd34c5b9e34440b7c3752ff1cdf4fe&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:43.918983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:43.933130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 361/710] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:43.934558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032768-c231-46bb-bf89-366b124945f6 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:43.934887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032768-c231-46bb-bf89-366b124945f6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:43.935172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032768-c231-46bb-bf89-366b124945f6 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:43.936996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 362/711] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:43.939539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47032768-c231-46bb-bf89-366b124945f6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:44.083820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28cc8-a8d7-4c8e-ad3a-4e0ac5e730da None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.086259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47032768-c231-46bb-bf89-366b124945f6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.092735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:44.125064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.125735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.126018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.126294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.126669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.127172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ac88daa9b11b4e13b89f3f981686ac72) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:44.127485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.127711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.128056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.128392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.128939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c8fa6-c3ef-4140-bfb3-f032b005cd3c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.129181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.130615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.135993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 363/712] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/role_assignments?user.id=ac88daa9b11b4e13b89f3f981686ac72&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:44.136561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.138625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.138625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:44.145753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:44.147072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:44.148284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 350/713] 10.222.0.64 () {56 vars in 1129 bytes} [Fri Jun 24 10:03:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:44.154347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:44.157211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:44.158930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:44.179470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:44.246924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 364/714] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:44.275277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:44.292426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7379022-1837-4e33-b334-34fd3e2157b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.340300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.341217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.341853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.342275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.343341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:44.343793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.345400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.357787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 351/715] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Jun 24 10:03:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:44.363646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:44.413505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2653-fa84-4e9c-9d4c-3271c47c2a99 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.418955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:44.457885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.459012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.459449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.459824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.460663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.462165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:44.479107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:44.480671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:44.481698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:44.486734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 365/716] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:44 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:44.490141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:44.507436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:44.600845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31caab-7b61-45e3-aff9-32f0cba2d792 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.604193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf844c-00ed-4e57-a163-9c84fd0f73a2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.640593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.641275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.641624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.641903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.642674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:44.643019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.644421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.649960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 366/717] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:03:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:44.658235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:44.676089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.676785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.677066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.677342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.678077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:44.678381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.679853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd6c2f-1ecc-4656-9c35-215f2f9a7312 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.693303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 352/718] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Jun 24 10:03:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:44.745632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:44.770843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9693ba-8f3d-44ff-953e-17f75e5a66b6 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:44.807568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:44.808281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:44.808566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:44.808846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:44.809589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] RBAC: Adding query filter params (type=network) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:44.809903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:44.811421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3789b-0448-473f-980b-fd32b5169573 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:44.818444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 367/719] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:03:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:46.065260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499d985-fe9c-4059-9b50-234b44dcf704 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.065964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499d985-fe9c-4059-9b50-234b44dcf704 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.065964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499d985-fe9c-4059-9b50-234b44dcf704 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.067401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 353/720] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:46.073216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.073854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.073854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.090760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:46.092071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:46.180843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 368/721] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:46.215995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499d985-fe9c-4059-9b50-234b44dcf704 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:46.372099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:46.380216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499d985-fe9c-4059-9b50-234b44dcf704 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:46.431448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b022f-5f34-4a3e-993e-00512803851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:46.432254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b022f-5f34-4a3e-993e-00512803851c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.432528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b022f-5f34-4a3e-993e-00512803851c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.432794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b022f-5f34-4a3e-993e-00512803851c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.452414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b022f-5f34-4a3e-993e-00512803851c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-808b022f-5f34-4a3e-993e-00512803851c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:46.460099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:46.461025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:46.462442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 354/722] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:46.467358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:46.495124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a8965-3151-439f-9869-c5ce2f617907 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.495449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a8965-3151-439f-9869-c5ce2f617907 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.495996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a8965-3151-439f-9869-c5ce2f617907 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.497868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 355/723] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:46.504343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.504625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.504899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.524477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:46.525678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:46.608730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:46.650737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e6e79-df0c-430c-96e7-cce53d1864d4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:46.656375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 356/724] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 10:03:46.701764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:46.708551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:46.710998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.711496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.712776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.713779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:46.718909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:46.720605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:46.735662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 369/725] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:46.741088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:46.867968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf40701-8463-473d-ba3b-e8ce269f1d6d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:46.888182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:46.903842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0aa5c-cd81-4771-a248-e66974e8b2d2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:46.908827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:46.917382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:46.918626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.919033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.919401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.934686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:46.936901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:46.942446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb5bde-5c50-4a23-81af-e49b8bc8bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:46.943525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb5bde-5c50-4a23-81af-e49b8bc8bf02 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.943970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb5bde-5c50-4a23-81af-e49b8bc8bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.944391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb5bde-5c50-4a23-81af-e49b8bc8bf02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.956045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb5bde-5c50-4a23-81af-e49b8bc8bf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-6e89c2d0-5ddf-43cd-b459-b48768289bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:46.959097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:46.960193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:46.961142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:46.961142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:46.961142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:46.961142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:46.961746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 370/726] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:46.963124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac17d-68ea-45c9-a182-1af1f4b3db86 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.963414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac17d-68ea-45c9-a182-1af1f4b3db86 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.963694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac17d-68ea-45c9-a182-1af1f4b3db86 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-14cb5bde-5c50-4a23-81af-e49b8bc8bf02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:46.967838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:46.968780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:46.969611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:46.970241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 371/727] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:46.970241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fac17d-68ea-45c9-a182-1af1f4b3db86 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:46.972505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 357/728] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:46.978849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:46.979754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:46.980234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.000925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:47.002581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:47.086573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 358/729] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:47.088260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.108320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fac17d-68ea-45c9-a182-1af1f4b3db86 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.154584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.155026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.155312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.155589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.156333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:47.156612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.158014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.171117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 372/730] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:47.172044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.173988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.201033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4690aef-7a89-4ec7-8aec-58234854ae82 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.217787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.239188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.240219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.240708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.241205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.242161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:47.242716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.244374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.254366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 359/731] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:47.259802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.280021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9db91e-77ab-42a9-bd75-576711bfd6c0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.316514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887233-549b-45df-aaea-9f346c67000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.317858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887233-549b-45df-aaea-9f346c67000f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.318296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887233-549b-45df-aaea-9f346c67000f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.318782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77887233-549b-45df-aaea-9f346c67000f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.329608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77887233-549b-45df-aaea-9f346c67000f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-77887233-549b-45df-aaea-9f346c67000f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:47.337213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:47.338145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:47.339032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:47.343535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 373/732] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:47.344433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887233-549b-45df-aaea-9f346c67000f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.373912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8676c93-0852-4e05-a106-621d46aba493 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.411489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.412224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.412506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.412782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.420900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.422368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.425271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:47.439421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:47.440470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 360/733] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:47.441399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.463099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77887233-549b-45df-aaea-9f346c67000f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.464766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.498247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.499077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.499353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.499622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.509932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.511435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-523079c1-dec3-4734-a791-357d4a6423ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:47.528143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:47.529308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:47.530264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:47.530264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:47.530264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:47.530264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:47.530264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 374/734] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:47.530881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f289e65-bdbf-4b33-b638-bba2c917c1ac None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.531212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f289e65-bdbf-4b33-b638-bba2c917c1ac None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.531464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f289e65-bdbf-4b33-b638-bba2c917c1ac None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.532696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 375/735] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:47.533807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf6842-dfb3-4f30-b84c-a3141f9fdc6e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.534090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf6842-dfb3-4f30-b84c-a3141f9fdc6e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.534362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf6842-dfb3-4f30-b84c-a3141f9fdc6e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.535600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 376/736] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:47.536292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf6842-dfb3-4f30-b84c-a3141f9fdc6e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.551341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2348bda-00b1-495e-8782-99a5dd7eb28a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.585142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.585813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.586096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.586375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.587159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:47.587469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.588857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.598320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 361/737] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:47.599244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.645228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf6842-dfb3-4f30-b84c-a3141f9fdc6e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.660726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.679519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.680120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.680398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.680671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.681522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:47.681828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.683320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ffb58-ffea-4c40-8eff-090f64e05c91 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.701496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 377/738] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:47.702446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb10b1e-42a3-44f1-8886-36e6ffb92312 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.702886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.703179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.703457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.721098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:47.722344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:47.730908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.744555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.745221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.745504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.745778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.746534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:47.746880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.748321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea6e87-cf40-42f6-9d7f-f29dc529a1d1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.760213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 362/739] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:47.762110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.762110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.762110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.778227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:47.779478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:47.795030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 378/740] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:47.796054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.859819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 363/741] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 10:03:47.860686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.895560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12105e4f-cf8b-4e7d-be70-206bc3f1b4f3 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.912048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.929343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.930205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.930439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.930738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:47.940806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:47.942273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:47.944114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:47.958592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46060a93-62ab-4b36-a2ff-5913365f6c77 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:47.960719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:47.961743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:47.962616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:47.962616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:47.962616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:47.962616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:47.962616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 379/742] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:47.962926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:47.994135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:47.994805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:47.995121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:47.995349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.062353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.062536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbec8b4-ad66-4712-93b4-8f430faed2f0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.063874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.095554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.096348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.096637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.096909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.103926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.105729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.114318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:48.120192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:48.121448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 380/743] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:48.122357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.167808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 364/744] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 10:03:48.168627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.176284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.220390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cef6d-aa24-4edd-bfc5-101d39bc7a97 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.256577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.257241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.257522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.257794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.258517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.259953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.271007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81877cb2-8a76-484a-b79c-7a08f22a997d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.275217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 381/745] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:03:48 2022] POST /identity/v3/services => generated 251 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:48.276021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.316289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.316933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.317210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.317479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.318208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.319642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.338116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 365/746] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:03:48 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:48.339452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.370170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.420830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fc593-a34a-4812-98da-80ca007e7a45 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.442118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.493452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.494104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.494370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.494647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.495451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:48.495951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.498242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.515115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 382/747] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:48.516203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.517955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5099-69a3-4734-b1bd-491c559add71 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.558201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.559055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.559796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.560191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.561386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:48.561774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.563515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.584028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 366/748] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:48.584840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.647004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36ab43-969f-4c90-a956-6fcb832711f7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.664053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.691858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.694805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.694805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.694805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.705532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee194-089b-460c-ba02-5f280b02b8e0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.709954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.711642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:48.734056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:48.735050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:48.735930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:48.735930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:48.735930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:48.735930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:48.736779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 383/749] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:48.738016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.747481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.748335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.748616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.748888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.757493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.759022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.772125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:48.776705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:48.777699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:48.778735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:48.779587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 367/750] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:48.780245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.858637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003941-3963-4110-84e6-0045cda8eb29 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.894289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.895006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.895286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.895559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.915285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d43110-9ad6-4159-97e8-e134dc18abd5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:48.943850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:48.954363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:48.955284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:48.955635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:48.956060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:48.956870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:48.957242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.958792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:48.972019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 368/751] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:48.973012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:48.978887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:48.980862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.011374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.102220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 384/752] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:03:49.103346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.109969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9a13d-700e-4bba-bb5a-b1e7e8af43fd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.149346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.150181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.150557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.150953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.151896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:49.152302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.153853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.173981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 369/753] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:03:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:49.175406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.226857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.249608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79e931-b3b6-411e-83d6-e1190a5d336a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.276774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.290517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.291458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.291840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.292210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.297749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef44a47-7e2d-4288-a93a-d87270e7963e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.299863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.301430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:49.315380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:49.316505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:49.317578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:49.323631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 385/754] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:49.323631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.348434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.349502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.350035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.350449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.363110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.364690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:49.384034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:49.385158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:49.386315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:49.386315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:49.386315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:49.386315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:49.392950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 370/755] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:49.394090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.443050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e265523-0194-4efa-952e-9c01064a2cc4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.487671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.488789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.489205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.489649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.501818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.503566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.509444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.517799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f4d81-73f7-4661-bc64-704e58fe3c94 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:49.523015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:49.524104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:49.525214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:49.525214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:49.525214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:49.525214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:49.531027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 386/756] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:49.532212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.559569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.560643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.560893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.561331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.562867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:49.563282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.565309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.571370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.586638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 371/757] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:49.586638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.651942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97355810-3a3d-47a5-84ce-871dec8c8e5d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.693338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.693338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.693338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.693338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.694341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:49.694761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.696236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.714785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 387/758] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:49.715849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.721609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b3a93-eb87-4199-bf76-252834e30af5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.760792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.761743 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.762229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.762728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.763674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:49.764156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.765748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.782347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 372/759] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:49.783734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.802802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.846650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:49.855409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052807-1b3f-4c97-a793-6cef0455c720 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.891166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.892202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.892662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.893164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.894193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=22d7019a89fb45c6afd1076f1ea120af) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:49.894730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.896397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.906769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc353e-93d7-41bf-b659-b9a542fa09c1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:49.907438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 388/760] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/role_assignments?user.id=22d7019a89fb45c6afd1076f1ea120af&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:49.908628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:49.946790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:49.947625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:49.948195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:49.948477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:49.957083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:49.959263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:49.976835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:49.977846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:49.978738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:49.983115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 373/761] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:49.986351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:50.081507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:50.110558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b30d-57e9-464f-b301-c06a4432cff8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:50.153007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:50.154485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.155415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.155929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.157044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:50.157630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:50.159282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:50.159843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ec266-3dcb-4c6e-8d49-46097cb04339 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:50.179441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 374/762] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:50.181443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:50.217816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:50.218950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.219410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.219742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.230006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:50.231327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:50.248705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:50.249690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:50.251055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:50.251835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 389/763] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:50.257150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:50.279385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:50.322929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03136bb4-7bdf-4717-92bd-9d0d0d5d19a1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:50.368262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e26b98-ae2a-412c-942b-9106325cf4a7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:50.369931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:50.371754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.372236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.372850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.373986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=19fd34c5b9e34440b7c3752ff1cdf4fe) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:50.374489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:50.376178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:50.380917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:50.391267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 375/764] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:50 2022] GET /identity/v3/role_assignments?user.id=19fd34c5b9e34440b7c3752ff1cdf4fe&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:50.427978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:50.428684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.428963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.429235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.430051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:50.430357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:50.431956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db984-3132-4cfb-b59c-bc2f9e1b6564 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:50.455440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 390/765] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:50.462517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:50.533628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5a5a6-d799-4086-b184-04498e9d3a3e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.534160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5a5a6-d799-4086-b184-04498e9d3a3e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.534583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5a5a6-d799-4086-b184-04498e9d3a3e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.537622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 391/766] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:50.541669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6688633-150d-4d91-bf49-d7e52af29e0f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.542370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6688633-150d-4d91-bf49-d7e52af29e0f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.542991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6688633-150d-4d91-bf49-d7e52af29e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.563486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b6688633-150d-4d91-bf49-d7e52af29e0f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:50.565541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b6688633-150d-4d91-bf49-d7e52af29e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:50.616027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbd7cb-fa4d-4ea1-bf41-7c29f9163b58 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:50.648703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 392/767] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:50.649823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67d851-6ab5-48e4-b847-931da83efb67 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.650114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67d851-6ab5-48e4-b847-931da83efb67 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.650391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67d851-6ab5-48e4-b847-931da83efb67 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.651957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 393/768] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:50.657700 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.657985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.658260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.668892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:50.669966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:50.670244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:50.670520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ac88daa9b11b4e13b89f3f981686ac72/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:50.672549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=ac88daa9b11b4e13b89f3f981686ac72, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:50.674018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:50.687918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:50.694744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 376/769] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:50 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ac88daa9b11b4e13b89f3f981686ac72/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:50.694744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:50.697766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:50.707661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:50.816496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:50.853354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 394/770] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:03:50.877408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:50.912482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d78b3-5062-4a86-b62d-d37f535ddbdc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.002451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d453-ac82-4b0d-9c5c-9acf1e4d6983 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.011429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.012268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.012551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.012825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] PATH_INFO: `/v3/services/network` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.013534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.014952 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:51.033002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:51.034043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:51.034909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 377/771] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:50 2022] GET /identity/v3/services/network => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:51.042698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.049185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.050320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.050855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.051334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.052229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.053803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:51.066661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:51.067642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:51.068474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:51.071826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 395/772] 10.222.0.64 () {56 vars in 1129 bytes} [Fri Jun 24 10:03:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:51.079684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.084762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:51.204325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788cbbb-acba-48bf-9b94-3f5204558c28 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.251411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:51.272352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.273033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.273365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.273831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.274547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:51.274885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.275003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8110-74ec-47f3-a8bd-fb0fc1d1b903 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.276282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.283420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 378/773] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:51.288706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.339757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.340455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.340727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.341006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.341973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:51.342264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.343998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.355200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 396/774] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:51.356422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.475482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:51.523358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa05a68-2691-4681-a24a-28a75914418a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.540317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010693c-160d-4bd2-914d-37bea9c161c5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.587837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.588517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.588800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.589076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.589791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:51.590098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.593211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.600249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.600943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.601229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.601513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.602235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:51.602541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.604022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.608509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 379/775] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:51.613708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 397/776] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:51.615179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.620159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.714990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:51.786853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064fcad-422d-4265-bdea-4a3939aa89e2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.801327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e758cf6-c119-496a-a769-e99fa4db0341 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.821775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.822800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.823363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.823809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.824810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] RBAC: Adding query filter params (interface=public, service_id=3de62cd5cfe24a12a2610b3a7fe14a6f, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:51.825278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.827078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.835449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 380/777] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/endpoints?service_id=3de62cd5cfe24a12a2610b3a7fe14a6f&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:03:51.840660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.845784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:51.854921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:51.855792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:51.856222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:51.857121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:51.858073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] RBAC: Adding query filter params (interface=public, service_id=d5150a47592e4dbd913709d4c5d05008, region_id=RegionOne) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:51.858516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:51.860068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:51.869178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 398/778] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/endpoints?service_id=d5150a47592e4dbd913709d4c5d05008&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:51.876026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:51.972728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed67fed-cad9-43fc-a71a-e99bb23c3835 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:51.983194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83501057-51a3-4fe2-8ee4-5cd6829eac19 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:52.018604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6d73-2718-4d9b-a3fc-6e9b8750fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:52.019560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6d73-2718-4d9b-a3fc-6e9b8750fb10 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.020018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6d73-2718-4d9b-a3fc-6e9b8750fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.020441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6d73-2718-4d9b-a3fc-6e9b8750fb10 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.021321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6d73-2718-4d9b-a3fc-6e9b8750fb10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:52.021962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:52.023103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6d73-2718-4d9b-a3fc-6e9b8750fb10 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:52.029334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 399/779] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:52.047749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe2073-1844-42ee-995b-7b8ecff7fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:52.048721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe2073-1844-42ee-995b-7b8ecff7fa21 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.049158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe2073-1844-42ee-995b-7b8ecff7fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.049586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe2073-1844-42ee-995b-7b8ecff7fa21 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.050544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe2073-1844-42ee-995b-7b8ecff7fa21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:52.052174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe2073-1844-42ee-995b-7b8ecff7fa21 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:52.064278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 381/780] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:03:51 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:52.506311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9d605-14e6-4f97-8d0a-4c7abbc2e576 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.507419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9d605-14e6-4f97-8d0a-4c7abbc2e576 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.507924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9d605-14e6-4f97-8d0a-4c7abbc2e576 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.509809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 400/781] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:52.515489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74a146-1378-458f-9836-1d88ac002c3d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.516501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74a146-1378-458f-9836-1d88ac002c3d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.516990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74a146-1378-458f-9836-1d88ac002c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.535034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6c74a146-1378-458f-9836-1d88ac002c3d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:52.536586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c74a146-1378-458f-9836-1d88ac002c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:52.631331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 382/782] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:52.669939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9d605-14e6-4f97-8d0a-4c7abbc2e576 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:52.672385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13cffd-9238-4de0-b65d-71504949642a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.672909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13cffd-9238-4de0-b65d-71504949642a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.673770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13cffd-9238-4de0-b65d-71504949642a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.676735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 383/783] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:52.682854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.683397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.683878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.705340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:52.707648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:52.774231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:52.798980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 384/784] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:03:52.826155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:52.863407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9d605-14e6-4f97-8d0a-4c7abbc2e576 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:52.876989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:52.942372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb09725-dc15-4bd2-849d-21df91e7b50e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:52.976384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:52.977225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.977513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.977816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.979896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:52.981192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:52.981672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:52.982224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:52.993693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:53.002540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:53.003689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:53.004518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:53.007328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 385/785] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:53.013759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.019365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:53.027266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:53.028204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:53.029079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:53.035164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 401/786] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:53.043777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.128351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e76de-0696-4dee-a91c-33f0c664f2d3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.139169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:53.167882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.168909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.169394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.169961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.171085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:53.171616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:53.173212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:53.183965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 386/787] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:53.189842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.301179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f8be4-53c7-487d-b426-209106e66416 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.306368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f612f3-8d0e-4814-b631-0bd4ed13feb1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.323080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:53.339348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.340115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.340336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.340620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.341597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:53.342760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:53.344861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d4685-c86d-41a5-9494-d8421d43ffe6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:53.355286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.356419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.356530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 402/788] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:53.356998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.357528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.358065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26160e-5f59-4dd5-ac7a-25206c19315f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.358390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26160e-5f59-4dd5-ac7a-25206c19315f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.358676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26160e-5f59-4dd5-ac7a-25206c19315f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.363979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 403/789] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:53.363979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26160e-5f59-4dd5-ac7a-25206c19315f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.374927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:53.376729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-cffa9d64-454c-484e-bb23-a261c54c9028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:53.396833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:53.397771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:53.398640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:53.398640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:53.398640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:53.398640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:53.400390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 387/790] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:53.406255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.406255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.406255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.427804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:53.429028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:53.451318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:53.485204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26160e-5f59-4dd5-ac7a-25206c19315f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.529324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.530455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 388/791] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:53.530933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.531014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.531086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.531086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.542465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:53.543992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:53.561676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:53.562582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:53.563491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:53.563491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:53.563491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:53.563491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:53.567732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 404/792] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:53.569277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.601800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:53.655040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4788648-60ee-42db-9f79-8a65f61134fe None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.691150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872a5c-3799-4e2a-aa5e-93ad939b7ed2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.729949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.731133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.731582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.732005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.744953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:53.753318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:53.754527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:53.755551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:53.757904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 405/793] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:53.758967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.827811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.828991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.829662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.830466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.831656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:53.832119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:53.834590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:53.850693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 389/794] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:53.852510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.857983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:53.867064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:53.877400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b4bd4-6c55-4763-b872-3a7d45250788 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:53.912688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:53.913395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:53.913760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:53.914039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:53.915064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:53.915510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:53.917063 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:53.933972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 406/795] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:53.934901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:53.977245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950513e-36b9-4ea8-af83-b0a6cc1f3443 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.014000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.014647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.014962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.015583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.016490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:54.016800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.018279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.027614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 390/796] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:54.028542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.062976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd892d2-1dc8-457a-ba94-7d9cea1956d4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.098503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.109759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.111963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.112395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.112816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.123486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.125172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.138166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e847-3dba-4d7b-8b47-39bc56f63386 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:54.145296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:54.146419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:54.147475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:54.149340 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 407/797] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:54.150814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.169766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.185113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.186064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.186437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.186645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.194721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.196192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:54.212293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:54.213317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:54.214276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 391/798] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:54.214921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.277682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c304-f399-43ad-a704-425e642b25bc None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.315095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.315937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.316218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.316532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.329446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.331951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.340434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e729b-e3e3-4b6e-8871-ff15c8e4af87 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:54.357058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:54.358098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:54.359033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:54.359033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:54.359033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:54.359033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:54.361468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 408/799] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:54.362917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.371553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.383342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.384167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.384594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.384968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.386486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:54.386609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.388431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.413132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 392/800] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:54.414288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.439921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.481331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442fa7-79c5-47fa-b3b7-c9295975d4b9 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.517168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.517852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.518373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.518515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.519336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:54.519586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.521038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.521128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a17c73-2634-4bd6-a0ab-9da3b3bd4f90 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.539081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 409/801] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:54.539655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.560054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.560767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.561050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.561354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.562085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:54.562393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.563785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14588-200a-47e9-a2db-06173e5e5849 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.575144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 393/802] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:54.576774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870312e7-1d52-4239-a637-a638efc161c4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.577064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870312e7-1d52-4239-a637-a638efc161c4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.577340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870312e7-1d52-4239-a637-a638efc161c4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.578765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 394/803] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:54.579757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f89146-cd0f-4ed0-8f4b-5e92393ece67 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.580157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f89146-cd0f-4ed0-8f4b-5e92393ece67 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.580289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f89146-cd0f-4ed0-8f4b-5e92393ece67 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.581412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 395/804] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:54.582097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f89146-cd0f-4ed0-8f4b-5e92393ece67 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.628931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.644915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f9e46-4b4c-46f7-820c-7125b49bbe59 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.666918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.678802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.679612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.679900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.680174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.683397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f89146-cd0f-4ed0-8f4b-5e92393ece67 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.690045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.691575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:54.709016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:54.710095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:54.711328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:54.711328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:54.711328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:54.711328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:54.711328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 410/805] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:54.711328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.722611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.723428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.723708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.723983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.734898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.736355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-bf9f5468-ae0c-42e5-804e-87fb4d22a350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:54.753302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:54.754374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:54.755488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:54.755488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:54.755488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:54.755488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:54.760237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 396/806] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:54.761516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec271f-9220-4a51-a71b-21a7165f309b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.761799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec271f-9220-4a51-a71b-21a7165f309b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.762068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec271f-9220-4a51-a71b-21a7165f309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.780548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-23ec271f-9220-4a51-a71b-21a7165f309b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:54.781761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-23ec271f-9220-4a51-a71b-21a7165f309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:54.815440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1004-d958-49ca-8066-9a0453528380 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:54.850724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:54.851519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.851677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 397/807] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:54.862889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.862889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.863059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:54.863059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:54.863059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:54.863317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:54.863317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:54.870602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:54.871817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:54.876326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:54.877284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:54.878202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 411/808] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:54.878812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.894916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.913583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:54.941279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 398/809] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:54.942193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:54.988388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41f34d-f4e7-438c-8000-d8ce52eccf81 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.021939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.022610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.022925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.023205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.023930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.024238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.025623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.036938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 412/810] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.037807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.040335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952375c-dcd5-49a2-b626-d7e22901965d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.076012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.076563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.076865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.077128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.077849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.078158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.079597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.090575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 399/811] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.091418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.109685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.133603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.141133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611e8ed-b370-4fe1-95f7-e04377247301 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.175118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.175841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.176122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.176394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] PATH_INFO: `/v3/services/network` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.177046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.178446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:55.190390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:55.191421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 10:03:55.192240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.193205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d655ed-f722-4cfd-aee9-ffe0797b2071 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.193287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 413/812] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:55.193287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.226023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.226685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.226995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.227263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.228090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.228392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.230180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.248386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 400/813] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.248954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.291411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236885-1c9e-465b-9728-9b83d84c95fe None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.322634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.324129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfc931-e422-4556-a933-727234711bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.324953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfc931-e422-4556-a933-727234711bba None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.325248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfc931-e422-4556-a933-727234711bba None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.325523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfc931-e422-4556-a933-727234711bba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.326409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfc931-e422-4556-a933-727234711bba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.327827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfc931-e422-4556-a933-727234711bba None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-d9bfc931-e422-4556-a933-727234711bba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:55.339778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:55.340738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.341627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 414/814] 10.222.0.64 () {56 vars in 1129 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:55.342565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfc931-e422-4556-a933-727234711bba None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.351373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b0e90-14f3-48fb-93f5-3689b321f000 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.370514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.386541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.387349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.387625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.387891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.394984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.396465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:55.410738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:55.411779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 401/815] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:55.412666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.447985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfc931-e422-4556-a933-727234711bba None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.480755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.481628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.481911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.482271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.489481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.490982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:55.505521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:55.506428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:55.507351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:55.507964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 415/816] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:55.508923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.512702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c33652-1c2d-47b0-88ef-3829e3900fff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.548523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.549182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.549466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.549747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.550451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.550788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.552180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.553262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.561891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 402/817] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.561891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.590951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.611311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7178a8-e135-4404-93c5-582652a29b1c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.645487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.646382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.646661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.646964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.665592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517106c0-159d-4917-9049-7780ed7727cc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.699926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.700522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.700818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.701051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.701759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.702065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.703477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.708379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 403/818] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.708753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.717554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.719058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.768669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.800174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.807630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e573d-85e1-4fb6-8069-896968ec8a69 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.819211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 416/819] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:03:55.820038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.841998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.842662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.842975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.843256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.844046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.844349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.845801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.857961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 404/820] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.858765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.920619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40dc67-9f7f-4f98-9ad5-22d66ecd21d7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.954448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.955060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.955342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.955618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.956401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.956508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b9519-cdfd-4123-a2a7-eafb08687f5e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:55.956707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.958141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:55.969454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 417/821] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:55.970245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:55.979668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:55.991652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:55.992315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:55.992590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:55.992861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:55.993875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:55.994180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:55.995588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.000562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 405/822] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:56.001334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.015907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:56.073532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3bd3a-d4b0-4920-ba50-a0f67dc7184e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.099324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1c3fb-48db-4290-b446-abe1374af4d0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.108833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.109508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.109787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.110062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.110809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:56.111117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.112520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.117856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 418/823] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Jun 24 10:03:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:56.118614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.134846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.135634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.135910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.136174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.147609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.149056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:56.165928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:56.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:56.167758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:56.167758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:56.167758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:56.167758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:56.167758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 406/824] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:56.168029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.208809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:56.223139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807a69f-0d75-4f05-a1e3-00f93638ddd1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.230371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:56.256832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.257733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.258012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.258285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/19fd34c5b9e34440b7c3752ff1cdf4fe/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.260287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=19fd34c5b9e34440b7c3752ff1cdf4fe, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.261735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.269212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86df235-32a4-4660-812c-c9f5e2f0a36f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.275870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 419/825] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:56 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/19fd34c5b9e34440b7c3752ff1cdf4fe/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:56.276641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.318877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.319891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.320172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.320444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/22d7019a89fb45c6afd1076f1ea120af/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.338470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=22d7019a89fb45c6afd1076f1ea120af, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.341653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.358537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 407/826] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:03:56 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/22d7019a89fb45c6afd1076f1ea120af/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:03:56.359285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.406570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0d730-4546-449c-a347-9b81a5db269a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.448137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.448910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.449196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.449478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.450259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.452028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.480797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 420/827] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:03:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:56.481585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.504883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:56.523297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1d6b-5879-4913-9700-d1f17610ecf2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.524298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:56.586777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.587570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.587853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.588177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.588898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.590321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.631669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 408/828] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:03:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:03:56.656351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161af0c8-8005-4b06-aff4-150060dcdc92 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.707780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.708509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.708797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.709135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.710242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:56.710242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.714882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.726583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 421/829] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:56.732235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:56.854532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23082f1-c016-487d-88d7-93d4b9256ab3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:56.892071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:56.892885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:56.893162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:56.893438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:56.901111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:56.902588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:56.906051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:03:56.919016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:56.920168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:03:56.921158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:56.923032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 409/830] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:03:56.926473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:57.017160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:57.040074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2f24-ef11-415e-906e-3b81740947be None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:57.075507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:57.076221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:57.076509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:57.076785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:57.077599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:57.077910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:57.079382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd9e21-bdcc-4673-9f33-9d9c87f09a97 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:57.091992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 422/831] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:57.097618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:57.199767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac301990-1b48-4f2d-861d-9e1b8f5ded33 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:57.237183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:57.237846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:57.238124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:57.238398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:57.239297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ac88daa9b11b4e13b89f3f981686ac72) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:57.239613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:57.241067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0df580-9b6f-464f-a032-979da976ad57 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:57.247735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 410/832] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:03:57 2022] GET /identity/v3/role_assignments?user.id=ac88daa9b11b4e13b89f3f981686ac72&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:58.450598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c4857-d224-418d-a386-6aaefd67466a None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.451891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c4857-d224-418d-a386-6aaefd67466a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.452393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c4857-d224-418d-a386-6aaefd67466a None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.454261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 423/833] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:58.460040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ba78a-30a1-49b9-b7db-cf047bc20b89 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.460923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ba78a-30a1-49b9-b7db-cf047bc20b89 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.461411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ba78a-30a1-49b9-b7db-cf047bc20b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.479572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-856ba78a-30a1-49b9-b7db-cf047bc20b89 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:58.481082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-856ba78a-30a1-49b9-b7db-cf047bc20b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:58.542513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:58.551769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 411/834] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:58.576664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c4857-d224-418d-a386-6aaefd67466a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:58.651710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc1dc-f514-4944-bdb8-6c68b48a3d6d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.652121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc1dc-f514-4944-bdb8-6c68b48a3d6d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.652300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc1dc-f514-4944-bdb8-6c68b48a3d6d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.654271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 412/835] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:03:58.660105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e377-d7e0-4806-8b26-d0112c76316e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.660386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e377-d7e0-4806-8b26-d0112c76316e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.660659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e377-d7e0-4806-8b26-d0112c76316e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.706810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c4857-d224-418d-a386-6aaefd67466a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:58.711385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-80c8e377-d7e0-4806-8b26-d0112c76316e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:58.713157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-80c8e377-d7e0-4806-8b26-d0112c76316e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:58.773020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:58.773843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.774131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.774405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.803401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:58.804538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:58.823541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:58.824617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:03:58.825505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:58.832847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 424/836] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:58.834457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:58.850727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 413/837] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:58.850727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e37b33-d753-4b75-a61a-0deb8e478918 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.850727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e37b33-d753-4b75-a61a-0deb8e478918 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.850727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e37b33-d753-4b75-a61a-0deb8e478918 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.852438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 414/838] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:58.853596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:58.853884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:58.854165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:58.877609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:58.878839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:58.959543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 415/839] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:58.960382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.031107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.079795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace44f4-8847-433c-a458-74235c23db98 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.093111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c61bb-2656-455b-a022-e99827d80946 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.117892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.118575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.118891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.119161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.119976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:59.120281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.121665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.130291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 425/840] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:03:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:59.130466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.131188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.132341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.133088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.133905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.146476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:03:59.154959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:59.155948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:59.156799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 416/841] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:03:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:59.158052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.200735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.243982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012104e-5097-4976-89dd-b03c05dbaae4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.267153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e7638-547e-48a2-9bfd-e9d149e78998 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.280695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.281510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.281789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.282080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.282644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.282779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.284118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:59.296452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:59.297521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:03:59.298458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 426/842] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:59.299910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.311703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.312813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.313165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.313511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.324658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.326146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:59.345292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:59.346444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:59.347394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:59.347394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:59.347394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:59.347394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:59.347394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 417/843] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:59.347394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.412181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f83b3-088f-40d7-8438-73f24f87ed18 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.436038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.447984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.448662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.448947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.449224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.449981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:59.450291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.452037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.453453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e03d2-0411-408c-9b90-f3f94f677653 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.459867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 427/844] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:59.460660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.489027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.489683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.489961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.490233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.490959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:59.491266 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.492635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77efc9-1ddd-4b65-b18d-5101fd999da9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.497153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 418/845] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:59.498874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1453f-1e3f-41d4-b54e-85c897d6e3b8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.499153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1453f-1e3f-41d4-b54e-85c897d6e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.499419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1453f-1e3f-41d4-b54e-85c897d6e3b8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.500806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 419/846] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:03:59.502885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1453f-1e3f-41d4-b54e-85c897d6e3b8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.533632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.563121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df135c-1cdb-40d0-838d-a6136fc2a032 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.597492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.598169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.598491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.598837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.599559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:59.599866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.601230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.604913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1453f-1e3f-41d4-b54e-85c897d6e3b8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.613515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 428/847] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:59.614309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.640693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.641602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.641892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.642170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.652412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.653859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.660170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0da9fae1-0301-497f-abf5-c61b20e0aad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:03:59.673060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:59.674170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:59.675238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:03:59.675238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:03:59.675238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:03:59.675238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:59.675238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 420/848] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:59.675531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.675682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.675945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.693041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:03:59.694209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:59.720595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eebea-7247-4334-ae29-2dca8ef06226 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.756656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.757255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.757537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.757823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.758536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:59.758901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.760265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.763270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 421/849] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:03:59.764192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.770886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 429/850] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:59.772148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.785657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.888147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6755642-ca9d-44f8-a19b-bd5b8a959917 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.911782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24b52e-8df2-489d-8ee2-c87e1481ce1c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:59.930770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.931295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:59.931885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.932178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.932432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.941980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.943575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:03:59.959315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:03:59.960499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 10:03:59.961827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:03:59.964333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:59.965309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:59.965309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:59.965531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 422/851] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:03:59.965858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:59.966240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:59.967395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:03:59.967802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:59.969271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:59.986316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 430/852] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:03:59.986316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.048946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.080250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9985b-842c-4eed-ae63-4b736a78f366 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.087692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab010c46-bdc8-46ea-8c75-969b622cdd55 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.117220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.118146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.118537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.118845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.119507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.120985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.123860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.124544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.124828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.125098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.125945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:00.126254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.127717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:00.133897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:00.134853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:00.135674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 423/853] 10.222.0.64 () {56 vars in 1132 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:00.137208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.150225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 431/854] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Jun 24 10:03:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:00.151195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.181338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.250166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524a079-88ef-4db5-a33c-ce20b86caa53 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.261987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7f4a5-04ca-448a-88c9-c032186137cd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.286229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.286896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.287178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.287454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.290543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.296347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.298062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.298182 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.298825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.299170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.299392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.300107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:00.300416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.302444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.306970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 432/855] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:00.307811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:00.315296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:00.316386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:00.317502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:00.321080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 424/856] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:00.322002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.417308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.427700 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a79c3-2160-41ef-9ca9-da5381ab5b1d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.432195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8c21-ec40-4cea-98ec-952cbdfe0316 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.464784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.465450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.465753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.466029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.467637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.468304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.468570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.468839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.469563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:00.469867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.471303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.487054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 425/857] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:00.487054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.540475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.541952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.557792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.606476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f77b6-9fd3-4faa-838a-e1258dc9da6d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.641934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.653070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.654274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.654849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.655384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.656490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:00.657011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.657739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 433/858] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:00.658855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.659416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.686281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 426/859] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:00.694477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.783926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dce815-b793-4f42-a70e-b09dc8a1bb88 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.824179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.825247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.825589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f8bd-481d-4c42-a2bb-03f7df66f2e5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:00.825891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.826321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.839844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.842180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.862926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:00.863792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:00.864174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:00.864549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:00.865806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:00.866876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:00.867984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:00.867984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:00.867984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:00.867984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:00.875006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 434/860] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:00.875208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.876587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:00.934840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:00.945772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:00.947320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:00.996305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc8bfd-b6bf-4cb9-9f48-169cff58c51f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.032597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.033720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.034237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.034797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.035729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:01.036252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.037918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.055313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 427/861] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:01.055850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 435/862] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:01.056715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.060005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.077858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.163008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32195bcd-4048-4d48-992c-1a99f03b8380 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.165222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b27dc-f8ec-4cc2-95e1-0b48b0db8414 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.167867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.199594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.200408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.200692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.201035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.204246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.205333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.205847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.206324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.208415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.210129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.218687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.220577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:01.224544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:01.225439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:01.226345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:01.231032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 428/863] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:01.232394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:01.243201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:01.244150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:01.245053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:01.245053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:01.245053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:01.245053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:01.247348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 436/864] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:01.253098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.331792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.358915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c5172-b69a-4292-ad57-dd626b53cf4b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.362969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acc42f-4162-4a7f-886f-ec6a6a1e6d62 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.396845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.397902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.398012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.398519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.398732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.398973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.399250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.399609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.399982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:01.400289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.401045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:01.401553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.401656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.403519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.413850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 437/865] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:01.420195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.426811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 429/866] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:01.428725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.440049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.526317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10c7d-e6b6-4745-9891-e74b2aac0591 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.539701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f04c2-f626-4227-925b-87bc1fe77519 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.561976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.563442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.563830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.564219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.573746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.575325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.578020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.579389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.579887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.580358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.581372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=19fd34c5b9e34440b7c3752ff1cdf4fe) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:01.581921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.583605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:01.593764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:01.594926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:01.595825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 438/868] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:01.596302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 430/868] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/role_assignments?user.id=19fd34c5b9e34440b7c3752ff1cdf4fe&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:01.604131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.710172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.727541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a673ef5-da0b-49bb-8ce1-819f8efd32df None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.767027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.767720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.768002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.768276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.769083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:01.769455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.771004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d7cec-97cc-46f7-9d4a-1a7597be243d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.784394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 439/869] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:01.790358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.793374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.802245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d4806-40db-40eb-96d6-65b9f18126b3 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.802545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d4806-40db-40eb-96d6-65b9f18126b3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.802854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d4806-40db-40eb-96d6-65b9f18126b3 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.804475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 440/870] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:01.809509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.809802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.810327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.828306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:01.829895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:01.911665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb04ba7-12eb-4aca-a06a-2d007a9f9bea None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.917391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 441/871] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:01.941178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.951189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.952110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.952398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.952677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.953628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=22d7019a89fb45c6afd1076f1ea120af) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:01.953945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.955432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542e5d1-9ded-44e8-b76a-2805d17f4bc6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.966967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 431/872] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/role_assignments?user.id=22d7019a89fb45c6afd1076f1ea120af&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:02.014160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:02.053790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd2ffa-a1d6-429a-abb9-b9128bea6fcb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:02.091634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69852a-4c94-4929-84c2-ced046eadd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:02.092334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69852a-4c94-4929-84c2-ced046eadd06 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:02.092618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69852a-4c94-4929-84c2-ced046eadd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:02.092895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69852a-4c94-4929-84c2-ced046eadd06 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:02.093659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69852a-4c94-4929-84c2-ced046eadd06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:02.095095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69852a-4c94-4929-84c2-ced046eadd06 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:02.114165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 442/873] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:01 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:02.631127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ffd68-340f-4e4f-91b6-ae1b3ed9e05f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:02.631684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ffd68-340f-4e4f-91b6-ae1b3ed9e05f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:02.631977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ffd68-340f-4e4f-91b6-ae1b3ed9e05f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:02.633724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 432/874] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:02.639065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bf424-e943-4e67-95e0-be0864c4f390 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:02.639467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bf424-e943-4e67-95e0-be0864c4f390 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:02.639657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bf424-e943-4e67-95e0-be0864c4f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:02.656887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-763bf424-e943-4e67-95e0-be0864c4f390 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:02.658083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-763bf424-e943-4e67-95e0-be0864c4f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:02.733648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 443/875] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:02.756924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ffd68-340f-4e4f-91b6-ae1b3ed9e05f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:02.831904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:02.870217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ffd68-340f-4e4f-91b6-ae1b3ed9e05f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:02.906156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ed8e4-0a03-4ca1-9890-cf249f7e97f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:02.906753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ed8e4-0a03-4ca1-9890-cf249f7e97f1 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:02.907036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ed8e4-0a03-4ca1-9890-cf249f7e97f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:02.907313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ed8e4-0a03-4ca1-9890-cf249f7e97f1 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:02.908045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ed8e4-0a03-4ca1-9890-cf249f7e97f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:02.909421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ed8e4-0a03-4ca1-9890-cf249f7e97f1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:02.927262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 433/876] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:02 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:03.757476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062d791-0fad-4e63-b27f-ced56223678c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:03.758478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062d791-0fad-4e63-b27f-ced56223678c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:03.759021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062d791-0fad-4e63-b27f-ced56223678c None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:03.760887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 444/877] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:03.766434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:03.767288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:03.767784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:03.785168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:03.786802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:03.864950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 434/878] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:03.887178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062d791-0fad-4e63-b27f-ced56223678c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:03.942041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:03.998733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062d791-0fad-4e63-b27f-ced56223678c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.033829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.034658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.034976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.035250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.035920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.037304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-110afe52-775e-45ae-bdb2-5c2deccc04f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:04.049928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:04.051022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:04.051855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 445/879] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:04.059714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.115212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.143995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b155d9-f10f-4f23-ac3d-b291186f028b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.144356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b155d9-f10f-4f23-ac3d-b291186f028b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.144562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b155d9-f10f-4f23-ac3d-b291186f028b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.146266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 446/880] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:04.150991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7609b7-9033-4032-a931-07c374f268c0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.151281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7609b7-9033-4032-a931-07c374f268c0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.151561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7609b7-9033-4032-a931-07c374f268c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.168035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427fc0c-bcf7-41e9-a81e-c1de3a27c786 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.169265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bd7609b7-9033-4032-a931-07c374f268c0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:04.170801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bd7609b7-9033-4032-a931-07c374f268c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:04.204711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.205742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.206246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.206746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.207734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:04.208233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.209963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.215507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 435/881] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:04.221389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.241330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 447/882] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:04.242685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5511f0-b9e1-40f8-a570-9b23b15dc737 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.243007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5511f0-b9e1-40f8-a570-9b23b15dc737 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.243280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5511f0-b9e1-40f8-a570-9b23b15dc737 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.246828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 448/883] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:04.250378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.250603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.252035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.273545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:04.275309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:04.320728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.332062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34defb03-8069-4111-9c9a-5f46fc3675d7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.358730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 449/884] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:04.358730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.364272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.378901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.379571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.379860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.380145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.380984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:04.381475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.382753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.387422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 436/885] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:04.395792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.484019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449557bd-71f8-48c6-b744-1187541c5f8b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.530619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c59fd-e088-40d0-a9eb-0862924a96a6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.536613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.537824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.538933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.539238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.554648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:04.563324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:04.564273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:04.565090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 450/886] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:04.571300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.581453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.581453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.581453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.581453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.582627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.583886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:04.597688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:04.598799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:04.599710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 437/887] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:04.606612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.618984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.660313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.696387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cb7d0-2a61-4789-ba23-0e48496d1bf7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.716579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2f32e-c1cb-4ed8-9abc-dc9330fc94d3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.736302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.737367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.737904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.738388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.739505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:04.740134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.741842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.752042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 451/888] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:04.752210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.752941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.753221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.753493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.754204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:04.754507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.761327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.761486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.761923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 438/889] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:04.767717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.876725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf6f35-ce6e-456b-9192-974be5c3f2de None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.887081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.889106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.893533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc2e6b-e184-42cb-8cae-2ecc8ae567be None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.918309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.919444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.920025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.920520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.921341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:04.921746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.923567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.934315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 439/890] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:04.934961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.935483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.936257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.936676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.937090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.951790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.954246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:04.980959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:04.981925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:04.983290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:04.983290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:04.983290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:04.983290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:04.983290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 452/891] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:04.989542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.047198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1a08b-b748-44e8-b85d-c378fba4bfe4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.084012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.084968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.085434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.085711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.086504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] RBAC: Adding query filter params (interface=public, service_id=fa9a5ccd03244e71af6e7d79dd256996, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:05.086843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.088500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b42c0-b433-40a7-a02c-80656577b336 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.098475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 440/892] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/endpoints?service_id=fa9a5ccd03244e71af6e7d79dd256996&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:05.098475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091c146-df1f-486c-abc9-6e75031572fc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.098475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091c146-df1f-486c-abc9-6e75031572fc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.098475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091c146-df1f-486c-abc9-6e75031572fc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.098475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 441/893] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:05.100065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091c146-df1f-486c-abc9-6e75031572fc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.103447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f3e4e-1d5e-4aca-89cb-7c37e5245b73 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.141492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.142079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.142362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.142636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.143173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.143422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:05.143707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.145379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887971a-57f9-47d4-aeee-92150c307bd6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.157800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 453/894] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:05.159152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.159447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.159725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.176400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:05.177596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.181253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.219945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091c146-df1f-486c-abc9-6e75031572fc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.259031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 454/895] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:05.259801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.264365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.265345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.265892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.266365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.267913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.270087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.277608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 442/896] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:04:05.278627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.390189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ca79f-72ab-423c-8c71-de29db793801 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.394197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.398296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc07e2-9ae1-4ae9-b003-99229a9574b0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.437287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.438903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.439643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.440583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.446058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.446910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.447196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.447476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.448143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.449534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.452120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.454541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.457471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:05.465399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:05.466468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:05.467424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 443/897] 10.222.0.64 () {56 vars in 1132 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:05.472137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:05.473394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:05.474355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 455/898] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:05.477376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.479595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.590434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c377b60-ce2d-4f82-945f-3a0516568f07 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.626543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.627286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.627572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.628040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.628559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:05.628867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.630256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.631854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e446a-088c-44bc-a503-43b579d1bb2d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.640179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 444/899] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:05.641567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.672274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.672908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.673263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.673612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.674571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:05.674939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.676684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.680134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.697938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 456/900] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:05.703867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.777443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9682f-fc62-4471-9950-fedb2efb1cbe None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.785351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.822821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.823595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.824093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.824464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.825559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:05.826342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.828107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.831658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3bf5-a1e6-47b4-b634-185e259f47c5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.835160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 445/901] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:05.843229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.878027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.879336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.880171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.880866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.949753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.962637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.963111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498c71-d126-494a-9eca-0b75a4758c2a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.964462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.007035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.008327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.008964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.009527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.010509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] RBAC: Adding query filter params (interface=public, service_id=51c0e591dd624219bc7ddf4206362bbd, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:06.011148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.013547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.028203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 446/902] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/endpoints?service_id=51c0e591dd624219bc7ddf4206362bbd&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:06.030216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.032209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.088976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 457/903] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 10:04:06.096451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.149231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb52a3-6159-43c9-b307-22b0f95cfb90 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.187455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.188537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.188920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.190167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.191006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.192508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.206829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 447/904] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:04:06.228171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b44848-1dff-453d-b171-f05442dc71d9 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.270925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.271756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.272042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.272320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.284713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.286217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:06.306074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:06.307060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:06.307951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:06.307951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:06.307951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:06.307951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:06.307951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 458/905] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:06.315931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.342881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.438663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06a1e-2503-4ace-8d53-069c9dbbaba8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.476971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.477714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.478001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.478276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.479042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:06.479710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.481105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09462f55-2617-4530-8fe8-7fc10f4df5be None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.494309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 448/906] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:06.499936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.512075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5c78-b84c-43ce-afc0-442ad5223870 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.512418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5c78-b84c-43ce-afc0-442ad5223870 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.512701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5c78-b84c-43ce-afc0-442ad5223870 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.514964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 449/907] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:06.519087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.519797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.520451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.520739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.543816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:06.546528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:06.626747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaeb69-c8c9-42c0-af14-eacb6a0a888d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.627158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 450/908] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:06.643598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.654286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.655070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.655357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.655627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.659415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.663820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.665266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:06.682749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:06.683697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:06.684608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 459/909] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:06.687418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.772326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c62e9b-f9c9-4cd0-8885-445a5536e051 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.783579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.816817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.818997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.819583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.820228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.821818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.823419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.823727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c1889-ccd8-4ae4-b7de-c3c722db8f36 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.849671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 451/910] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:06.875282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.876302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.876766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.877263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.878366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:06.878904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.880690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fade2-efe8-43c2-a7c1-4defc0ff082c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.896366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 460/911] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:06.906767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.987247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:07.033346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2e48d-5c9e-4065-a700-481335812b73 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.068881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.069545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.069823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.070093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.070956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] RBAC: Adding query filter params (role.id=e2c83d048daa420bb9f1204899da2381, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=22d7019a89fb45c6afd1076f1ea120af) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:07.071260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.072703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b002b6-cc36-4bca-a92b-7fdde1edb7fd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.079832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 452/912] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/role_assignments?user.id=22d7019a89fb45c6afd1076f1ea120af&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=e2c83d048daa420bb9f1204899da2381 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:07.391466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9dc10-f18e-42b1-933c-0068485c4951 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.392232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9dc10-f18e-42b1-933c-0068485c4951 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.392716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9dc10-f18e-42b1-933c-0068485c4951 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.394574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 461/913] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:07.399781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.400072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.400344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.418115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:07.419344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:07.493077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 453/914] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:07.515771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9dc10-f18e-42b1-933c-0068485c4951 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.621892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9dc10-f18e-42b1-933c-0068485c4951 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.649452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:07.660021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.660884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.661165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.661439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.662148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.663604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:07.676043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:07.677114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:07.678619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 462/915] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:07.683185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.763349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:07.803002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1c580-8157-457d-a1e3-14c4b5d2edca None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.838348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.839895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.840399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.840886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.841870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:07.842389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.844073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.849626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 454/916] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:07.855423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.959793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc53361-0dc8-4a33-a814-9923e72ccdb2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.993539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.994215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.994494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.994792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.995511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:07.995816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.997192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b736a-7e01-442d-a60a-a86b619cc706 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.002858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 463/917] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:08.008594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.117798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a314c-1228-411b-8272-c17d3ce97fd7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.145129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:08.153520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1faa165-adb7-4ce9-bd06-30bad4c9413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.154751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faa165-adb7-4ce9-bd06-30bad4c9413f None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.155449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faa165-adb7-4ce9-bd06-30bad4c9413f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.155724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faa165-adb7-4ce9-bd06-30bad4c9413f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.156511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1faa165-adb7-4ce9-bd06-30bad4c9413f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.162915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1faa165-adb7-4ce9-bd06-30bad4c9413f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.180368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 455/918] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:04:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:08.403665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0baef-fd94-4ab1-bee7-13723f0b5b8c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.404166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0baef-fd94-4ab1-bee7-13723f0b5b8c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.404531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0baef-fd94-4ab1-bee7-13723f0b5b8c None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.405887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 464/919] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:08.411375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.411901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.412378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.430344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:08.431900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:08.507802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 456/920] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:08.532105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0baef-fd94-4ab1-bee7-13723f0b5b8c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.594445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:08.643327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0baef-fd94-4ab1-bee7-13723f0b5b8c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.678626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.679716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.680145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.680582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.681410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.682971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:08.710745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:08.711798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:08.712614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 465/921] 10.222.0.64 () {56 vars in 1132 bytes} [Fri Jun 24 10:04:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:08.718157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.829637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2e57a-d0b6-437a-a440-2a39747fe06a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.840870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:08.867850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.868528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.868811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.869085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.869810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:08.870115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.871537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23484176-d324-49c8-b3ee-0089016bc810 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.877126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 457/922] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:08.882573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.985779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3380ca1-cc2c-4381-ac8f-9aac27997c2b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.003560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.014915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed3f0-2dd4-4c2a-8aa9-ce02061cb60c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.015541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed3f0-2dd4-4c2a-8aa9-ce02061cb60c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.016048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed3f0-2dd4-4c2a-8aa9-ce02061cb60c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.018493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 458/923] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:09.022390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.022390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.022390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.022922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.023321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:09.023630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.024770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.025012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.025490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.025969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.030604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 466/924] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:09.035744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.044923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:09.046515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.135070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 459/925] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:09.152759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295bd15-13c4-4031-9fe7-6a098f352bec None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.157245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.188925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818186d5-8bbe-4dbc-965a-b14b1890c917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.189601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818186d5-8bbe-4dbc-965a-b14b1890c917 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.189891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818186d5-8bbe-4dbc-965a-b14b1890c917 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.190174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818186d5-8bbe-4dbc-965a-b14b1890c917 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.191547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818186d5-8bbe-4dbc-965a-b14b1890c917 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.193088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818186d5-8bbe-4dbc-965a-b14b1890c917 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.220591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 467/926] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:09.252858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9cfdb-fc48-4dd2-8cfd-333e04005880 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.253212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9cfdb-fc48-4dd2-8cfd-333e04005880 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.253615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9cfdb-fc48-4dd2-8cfd-333e04005880 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.259503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 468/927] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:09.261338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.261745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.262127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.285995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:09.287718 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.296340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ff92-0868-4f92-85fb-4e779681edfb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.299535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.327544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.356852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.357845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.358129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.358406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.359128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.360611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:09.374086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:09.375245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:09.376088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:09.380740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 460/928] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:09.381730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.398746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 469/929] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:09.415085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.511923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571c8c-e308-4ea3-b2af-94268ebf6163 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.541032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7eed7-7d4c-49fc-99b1-ea0ebfc34e83 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.558385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.559942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.560583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.561072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.561950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:09.562409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.564902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.572970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 461/930] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:09.579207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.581966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.582951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.583325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.584394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.595049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:09.603121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:09.604658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:09.605697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:09.608553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 470/931] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:09.614412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.617490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.645291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.715734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67de0f0-7eb8-4746-a72a-2266f86f4272 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.735951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25858e-14a2-485c-990b-9a4eb0024047 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.757671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.758395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.758817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.759194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.760013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:09.760476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.762166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.774095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 462/932] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:09.775248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.783945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.785464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.785973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.786465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.787831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:09.788666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.790614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.805177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 471/933] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:09.807765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.881589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.893220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96ef12-3ef5-4272-a1f8-df93bd67246a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.904401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.914813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39882-f247-40d5-bb71-1a3e3b849c4c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.932914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.933927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.934414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.934917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.936136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] RBAC: Adding query filter params (interface=public, service_id=75a620664c464c5f88403873938a0c26, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:09.936672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.938277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.944423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 463/934] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/endpoints?service_id=75a620664c464c5f88403873938a0c26&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:09.949977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.954338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.955464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.955960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.956429 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.969159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.971548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:09.995222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:09.996204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:09.997197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:09.997197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:09.997197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:09.997197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:09.997197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 472/935] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:10.002893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.065566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7165ef-021c-42c0-911f-b4a8a93ebfcc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.101631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.102841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.103383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.103876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.105218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.106946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.114211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 464/936] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:04:10.122887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158e32a-0327-4208-91df-8112868712d7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.156386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.157075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.157359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.157637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.158378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:10.158690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.160123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.178862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 473/937] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:10.180340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.208367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.295161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976197fc-f73d-4050-93bd-0bda0b31e983 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.335190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.335983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.336269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.336547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.345071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.346546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ad63a7b6-2587-4868-980f-96bdc17aa587 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:10.362370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:10.363577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.364839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 465/938] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:10.371386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.379050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.387901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6fe6c-7317-4420-8e0d-f8260dee2ffc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.387901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6fe6c-7317-4420-8e0d-f8260dee2ffc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.387901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6fe6c-7317-4420-8e0d-f8260dee2ffc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.390491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 466/939] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:10.396686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.396960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.397249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.421346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:10.422596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:10.490087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420b07a-e364-4e90-bf41-bf272505e83d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.514801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 467/940] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:10.533734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.535644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.537788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.538084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.539995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.540179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:10.542237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.544038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.549851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.574931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 474/941] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:10.578050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.649750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.677631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9024a-3e88-4f46-b06e-5daf80a6a632 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.703539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c86fae-91b4-4726-93c4-1474ab2a6a46 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.715367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.716044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.716325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.716815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.717702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.719043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.720442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING keystone.server.flask.common [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 10:04:10.739073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 10:04:10.739073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 10:04:10.739073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [parameters: {'id': 'c101c13f0c044486b746f9b7504f6113', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 10:04:10.739073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 10:04:10.748455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c101c13f0c044486b746f9b7504f6113.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c101c13f0c044486b746f9b7504f6113. Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:04:10.748922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 10:04:10.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() Jun 24 10:04:10.750693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 10:04:10.751612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 10:04:10.753230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:04:10.755206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'id': 'c101c13f0c044486b746f9b7504f6113', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.756314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:10.757348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c101c13f0c044486b746f9b7504f6113. Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.758500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 468/942] 10.222.0.64 () {60 vars in 1140 bytes} [Fri Jun 24 10:04:10 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:10.759848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.760059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.760301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.760573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.773775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.775571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:10.795138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:10.796109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:10.796996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:10.796996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:10.796996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:10.796996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:10.801102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 475/943] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:10.802135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.845845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.888337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32577d-ba7e-4241-ad0b-3b428e82d2de None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.916956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0b476-18b9-4b43-98e8-1f844ad5c34a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.931074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.931458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.933372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.933969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.935206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.945823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.948768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.954337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.955078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.955357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.955632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.956363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:10.956670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.958032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:10.968979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:10.969903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:10.970893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:10.978826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 476/944] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:10.978826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.979013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 469/945] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:10.983797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.102728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4625eb-eeaf-42e0-96b2-5e31ace8e7cd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.103290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48757951-3efa-4ec2-be8e-a9fd7ef88a2b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.132374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:11.143053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.143711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.143993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.144268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.145031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:11.145335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.146811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.159206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.160328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.160820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.161875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.174817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 470/946] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:11.176436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.177941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b0cf296e-d39c-4fc3-adf0-a2e78b0adc64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:11.194119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:11.195147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:11.197229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 477/947] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:11.201691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.319770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9897654-7da0-4885-afa1-f2a72a4abf49 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.360133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.361224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.361719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.362214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.363361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:11.363882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.365582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.370911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707c29-2c7d-410c-a5fe-9314d71495a5 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.371206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707c29-2c7d-410c-a5fe-9314d71495a5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.371483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707c29-2c7d-410c-a5fe-9314d71495a5 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.373370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 478/948] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:11.378861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.379065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.379341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.380989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 471/949] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:11.387427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.401759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:11.402973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:11.436744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:11.448640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:11.498678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 479/950] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:11.519361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc073-3f36-44ba-a3a0-c01b0b3fa58f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.522193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.554382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.555315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.555596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.555871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/22d7019a89fb45c6afd1076f1ea120af/roles/e2c83d048daa420bb9f1204899da2381` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.558256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=22d7019a89fb45c6afd1076f1ea120af, role_id=e2c83d048daa420bb9f1204899da2381)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.559748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.574824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 472/951] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:04:11 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/22d7019a89fb45c6afd1076f1ea120af/roles/e2c83d048daa420bb9f1204899da2381 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:11.636668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f2532-a552-4958-92bf-b6baa7913d1c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.676416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.677403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.677763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.678112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.678891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.680453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:11.693995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:11.695259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:11.696256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:11.702744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 480/952] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:11.704858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.826349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f89ed1-5442-4a37-b908-fbde7854c5d6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.834191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:11.864178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.865142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.865566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.865981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.866870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:11.867321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.868870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f52ee-a33e-4311-8c79-bca90e81f042 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.879065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 473/953] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:11.880926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.912065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:11.988270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a846d3-a1ab-4804-adf6-390422af6679 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:12.023639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:12.024657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:12.025084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:12.025508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:12.026376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:12.026870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:12.028388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383faa65-a7da-450d-8632-5687ae9024e1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:12.033938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 481/954] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:12.298399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e666741-3a8d-49ac-a6fe-aec1634a168d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:12.298970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e666741-3a8d-49ac-a6fe-aec1634a168d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:12.299167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e666741-3a8d-49ac-a6fe-aec1634a168d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:12.300997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 474/955] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:12.306360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:12.306652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:12.306955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:12.328579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:12.330159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:12.402301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 482/956] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:12.426932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e666741-3a8d-49ac-a6fe-aec1634a168d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:12.537961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e666741-3a8d-49ac-a6fe-aec1634a168d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:12.572762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:12.573492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:12.573775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:12.574050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:12.574720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:12.576184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:12.588469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:12.589577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:12.590735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 475/957] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:12.598611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:12.620965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:12.699331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dedfe-d6b0-4bf1-9a99-95bbecbf125d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:12.734274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:12.734982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:12.735272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:12.735540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:12.736254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:12.736559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:12.737955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:12.743975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 483/958] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:12.750456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:12.786215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:12.859035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0ec2b-7f9f-4820-be77-5ed7173d0bb8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:12.893889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:12.894558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:12.894869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:12.895146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:12.895863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:12.896184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:12.897609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5947a-c30d-42ca-b2c0-e4fcebcef602 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:12.903009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 476/959] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:12.909025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:12.993181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:13.009989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a89309-aa9e-4bbd-b7e4-bf04012bf8bf None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:13.044879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44d28-437c-473e-919e-1f8743ca919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:13.045550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44d28-437c-473e-919e-1f8743ca919f None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.045828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44d28-437c-473e-919e-1f8743ca919f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.046101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44d28-437c-473e-919e-1f8743ca919f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.046888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a44d28-437c-473e-919e-1f8743ca919f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:13.048265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a44d28-437c-473e-919e-1f8743ca919f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:13.069805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 484/960] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:04:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:13.469379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ffb6e-1d15-4a16-8876-12968a48d827 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.469962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ffb6e-1d15-4a16-8876-12968a48d827 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.470231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ffb6e-1d15-4a16-8876-12968a48d827 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.472034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 477/961] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:13.477328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733c96-a3da-44b0-9bd5-3c7777bc8226 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.477724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733c96-a3da-44b0-9bd5-3c7777bc8226 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.478102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733c96-a3da-44b0-9bd5-3c7777bc8226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.495753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-34733c96-a3da-44b0-9bd5-3c7777bc8226 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:13.496960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-34733c96-a3da-44b0-9bd5-3c7777bc8226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:13.573353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 485/962] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:13.598264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ffb6e-1d15-4a16-8876-12968a48d827 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:13.692937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0729-a80c-44ab-88a9-eb5b0082f9d7 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.693283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0729-a80c-44ab-88a9-eb5b0082f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.693491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0729-a80c-44ab-88a9-eb5b0082f9d7 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.695150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 486/963] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:13.700625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.700957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.701859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.705565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:13.712012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ffb6e-1d15-4a16-8876-12968a48d827 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:13.723344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:13.724464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:13.749610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:13.750762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.751253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.751707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.752696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:13.754377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:13.770047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 10:04:13.770047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 10:04:13.770047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [parameters: {'user_id': '4c232a7615a64426b19729a7fdb585b6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 10:04:13.770047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jun 24 10:04:13.779094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:04:13.780000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 10:04:13.780977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 10:04:13.781910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 10:04:13.783108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 10:04:13.783957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'user_id': '4c232a7615a64426b19729a7fdb585b6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:13.784791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 10:04:13.785622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 10:04:13.786601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 10:04:13.786601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 10:04:13.786601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 10:04:13.786601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 10:04:13.786601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 10:04:13.786601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:13.789276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:13.789725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 478/964] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Jun 24 10:04:13 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:13.796628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:13.801869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 487/965] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:13.827297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:13.912206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f1ab4-f259-4ad3-b938-d0144e10331a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:13.936473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8deb6b-51f5-42e0-8687-9c2fbfe2f84d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:13.951041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:13.952173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.952622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.953044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.961521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:13.963512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:13.972304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:13.973265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.973716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.974031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:13.981211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:13.982449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 10:04:13.983550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:13.986006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 479/966] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:13.991333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:13.991703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:14.000309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:14.001916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:14.003061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 488/967] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Jun 24 10:04:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:14.007203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:14.019015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:14.065511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:14.110497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d19191-9618-4d99-bcb0-fa5e5df00c09 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.111077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9968-650d-45a0-b548-30fb12af6e50 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.146015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.146766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.147051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.147325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.148127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:14.148435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.149810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.150914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.151991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.152476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.152971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.154036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:14.154770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.156495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b08e91-368f-4ad3-91c4-8ef75f8c1ca6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.158675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 489/968] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:04:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:14.163746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:14.198933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 480/969] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Jun 24 10:04:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:14.201643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a0a94-dcf9-4009-bb2f-1ff915c37dba None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.201902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a0a94-dcf9-4009-bb2f-1ff915c37dba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.202210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a0a94-dcf9-4009-bb2f-1ff915c37dba None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.203815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 481/970] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:14.209666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.210004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.210195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.230328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:14.232113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:14.319803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db918ee4-d474-463d-80fd-83b91e53a404 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.320558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 482/971] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:14.339053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:14.339880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:14.351481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:14.366810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.367573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.367856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.368130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.381298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.386695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:14.407666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:14.408616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:14.409509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:14.409509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:14.409509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:14.409509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:14.409509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 490/972] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:14.415746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:14.463196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58f14-29b6-42c0-9a02-14c79a06ff42 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.504715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.505355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.505577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.505852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.506641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.508207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.526112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 483/973] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:14 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:14.542272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8042a9e-2cce-413e-a627-7d932810e8fd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.593257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.593947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.594242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.594497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.595318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:14.595623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.597037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.616913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 491/974] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:14.618808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:14.701148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:14.738399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788e2da-cf66-498c-849d-7d569380b4e9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.773395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.774632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.775080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.775517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.783878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.785516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:14.800427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:14.801505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 10:04:14.802435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:14.803031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 484/975] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:04:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:14.808301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:14.836784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:14.915250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257b5e4-8034-4e41-bb6e-f96a30a5b073 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:14.950166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:14.950908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:14.951202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:14.951476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:14.952327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:14.952637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:14.954087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03126115-984b-4892-ba7f-b1343134a6e2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:14.972549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 492/976] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Jun 24 10:04:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:14.978150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.084971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f8b49-c905-451d-a349-52c298e6456a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.112532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.121435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.122095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.122375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.122730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.173917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe3a7e-bdb0-4bbc-b3cb-021e79900cc8 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.174492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe3a7e-bdb0-4bbc-b3cb-021e79900cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.174492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe3a7e-bdb0-4bbc-b3cb-021e79900cc8 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.176031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 493/977] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:15.181074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.181370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.181656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.196896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.198270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:15.198885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.199502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:15.276866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 494/978] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:15.299964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.310332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 485/979] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:14 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:15.311759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.316832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.342458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.409682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a052b64-fe75-4c11-9a1f-da053cf9f88f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.426875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffc924-55ca-4896-ac26-39f5d516ff58 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.444853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.445670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.445951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.446224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.446936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.448319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:15.460380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:15.461364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:15.463020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 495/980] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:15.467897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.468199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.469002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.469283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.469553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.480501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.482249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:15.499876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:15.500779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:15.501649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:15.501649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:15.501649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:15.501649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:15.501649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 486/981] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:15.506581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.552051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.576838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b7a60-5575-4e70-8ba8-7eae832d3388 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.604508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.613094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.613775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.614367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.614367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.615056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:15.615421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.616833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.623934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 496/982] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:15.629585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.632085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c918-aed6-4cea-b6a2-b5a5edfdaac6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.677584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.678248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.678678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.678989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.680143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:15.680722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.682299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.698783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 487/983] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:15.703451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.750958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857060b6-c498-48e4-8ce8-3ae436447e5d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.785594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.786256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.786541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.786840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.787545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:15.787849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.789215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.794229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 497/984] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:15.800787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.813091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.815829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf89da-fe5c-48aa-8a31-1d8f64747c36 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.852192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.853079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.853366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.853642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.861105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.862510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.865672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:15.879842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:15.880766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:15.882384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 488/985] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:15.886737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.921973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a64c4-7299-4a4a-a548-10ad75f73fca None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:15.958210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:15.959082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:15.959366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:15.959639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:15.960463 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] RBAC: Adding query filter params (interface=admin, service_id=75a620664c464c5f88403873938a0c26, region_id=RegionOne) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:15.960785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:15.962212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:15.968678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 498/986] 10.222.0.64 () {56 vars in 1249 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/endpoints?service_id=75a620664c464c5f88403873938a0c26&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:15.974282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:15.994178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7867-6bc5-4c80-92f6-38bfb6af6b01 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:16.030739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:16.031414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:16.031695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:16.031967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:16.032754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:16.033060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:16.034494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:16.046395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 489/987] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:16.051067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:16.052148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:16.086292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b012c-0572-45d5-bf57-ef6a9b006d47 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:16.114582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:16.130218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7f5b5-9e95-485b-82bb-970b99ffd22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:16.130926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7f5b5-9e95-485b-82bb-970b99ffd22b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:16.131219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7f5b5-9e95-485b-82bb-970b99ffd22b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:16.131480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7f5b5-9e95-485b-82bb-970b99ffd22b None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:16.132680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7f5b5-9e95-485b-82bb-970b99ffd22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:16.134519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7f5b5-9e95-485b-82bb-970b99ffd22b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:16.154926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 499/988] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:04:15 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:16.161346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002685a8-d519-4155-8517-e657aa76f8a3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:16.202933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:16.203620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:16.203898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:16.204160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:16.205011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] RBAC: Adding query filter params (role.id=e2c83d048daa420bb9f1204899da2381, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=22d7019a89fb45c6afd1076f1ea120af) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:16.205317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:16.206812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a729d-50d4-4c89-9c46-e264c2a4a0b0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:16.214238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 490/989] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:16 2022] GET /identity/v3/role_assignments?user.id=22d7019a89fb45c6afd1076f1ea120af&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=e2c83d048daa420bb9f1204899da2381 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:16.598249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda448-7edf-4d63-a415-29908b466b9f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:16.598817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda448-7edf-4d63-a415-29908b466b9f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:16.598817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda448-7edf-4d63-a415-29908b466b9f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:16.600923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 500/990] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:16.606683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:16.607140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:16.607336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:16.626553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:16.627796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:16.706786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 491/991] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:16.726475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dda448-7edf-4d63-a415-29908b466b9f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:16.836087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dda448-7edf-4d63-a415-29908b466b9f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:16.872847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:16.873709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:16.873992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:16.874272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:16.875039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:16.876419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:16.878519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:16.889935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:16.891000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:16.891847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:16.894883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 501/992] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:16.897264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:16.957501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:17.005746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38ff5c-7d58-4638-b1f3-72314ef589f0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:17.041976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:17.042636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:17.042944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:17.043213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:17.043919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:17.044222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:17.045704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:17.050853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 492/993] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:17.056634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:17.156491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fded8-c085-41a6-b19a-6281965690c6 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:17.189400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:17.190073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:17.190355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:17.190628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:17.191367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:17.191673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:17.193045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:17.198094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 502/994] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:17.204075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:17.312894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a9337-8847-4e81-ae68-64d011c3c10f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:17.331020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:17.349600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:17.350267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:17.350549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:17.350856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:17.351648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] RBAC: Adding query filter params (interface=public, service_id=603f81a452084e46a6a646fc0e51b3db, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:17.351958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:17.353359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f81ba-66c0-442e-8e56-2b0f57741ee3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:17.363552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 493/995] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:04:17 2022] GET /identity/v3/endpoints?service_id=603f81a452084e46a6a646fc0e51b3db&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:17.364759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:17.435056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:17.475311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc8f80-27aa-4311-9f66-d02226ad3205 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:17.508894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550b039-0d0f-41a0-9cfa-32c4dc19f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:17.509567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550b039-0d0f-41a0-9cfa-32c4dc19f16f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:17.509850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550b039-0d0f-41a0-9cfa-32c4dc19f16f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:17.510125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550b039-0d0f-41a0-9cfa-32c4dc19f16f None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:17.510841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550b039-0d0f-41a0-9cfa-32c4dc19f16f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:17.512225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550b039-0d0f-41a0-9cfa-32c4dc19f16f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:17.518771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 503/996] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:04:17 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:18.184579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b3d1-f4f5-4b9e-afe4-327b34ec1143 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.185272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b3d1-f4f5-4b9e-afe4-327b34ec1143 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.185272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b3d1-f4f5-4b9e-afe4-327b34ec1143 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.187671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 494/997] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:18.192311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.192781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.192862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.210432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:18.211697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:18.286530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 504/998] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:18.308376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2b53a-9497-45d9-9245-c9aff0cbe7dc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.308965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2b53a-9497-45d9-9245-c9aff0cbe7dc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.309337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2b53a-9497-45d9-9245-c9aff0cbe7dc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.309498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.311059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 495/999] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:18.316039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.316439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.316721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.335318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:18.336529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:18.372323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:18.410805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 496/1000] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:18.421134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5f43-0712-445f-a176-d8c1ddd7142b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.433386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.459518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.460254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.460536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.460808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.461456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.462876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:18.474846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:18.475862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 10:04:18.476724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:18.480683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 505/1001] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Jun 24 10:04:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:18.482188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.540197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8295d9-261f-4915-9652-4540d38c52ad None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.554635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:18.577602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.578878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.579221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.579469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] PATH_INFO: `/v3/services/image` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.580087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.581532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.590143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b740-be6c-44f0-824f-c2c4903f141a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:18.593482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:18.594770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:18.595661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 497/1002] 10.222.0.64 () {56 vars in 1105 bytes} [Fri Jun 24 10:04:18 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:18.600880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.619454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:18.635574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.636249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.636598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.636865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.637488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:18.637790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.639323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.644446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 506/1003] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:18.649779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.709669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daf09-829d-4ebe-9e2c-f5cdac52b678 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.751737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.752411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.752837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.753113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.753973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:18.754235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.755648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.761131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 498/1004] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:18.761578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c6e69-04b0-4e5d-a430-b2e2daeaeae4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.766805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.804747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.805413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.805688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.805959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.806659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:18.806983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.808348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.811229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:18.819025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 507/1005] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Jun 24 10:04:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:18.820142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:18.872380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:18.880582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fdd93-12fb-4f02-921d-160190dc5ac2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.914411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.915106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.915387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.915663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.916382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:18.916682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.918074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d32228-8074-46f6-bfd7-65e0755c97f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.922778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 499/1006] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:18.934989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e08950-8ad1-44af-b041-a2a6a00fdf7a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:18.964107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0788f-f2dd-4558-aa6b-41891911b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:18.964799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0788f-f2dd-4558-aa6b-41891911b380 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:18.965150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0788f-f2dd-4558-aa6b-41891911b380 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:18.965506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0788f-f2dd-4558-aa6b-41891911b380 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:18.966326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0788f-f2dd-4558-aa6b-41891911b380 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:18.968033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0788f-f2dd-4558-aa6b-41891911b380 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:18.995036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 508/1007] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:04:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:19.559452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812bbd-1d18-46ef-88c8-6ac13293eade None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:19.560015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812bbd-1d18-46ef-88c8-6ac13293eade None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:19.560015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812bbd-1d18-46ef-88c8-6ac13293eade None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:19.561874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 500/1008] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:19.567382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:19.567820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:19.568010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:19.585726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:19.586967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:19.664291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 509/1009] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:19.687570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812bbd-1d18-46ef-88c8-6ac13293eade None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:19.799221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812bbd-1d18-46ef-88c8-6ac13293eade None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:19.802133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:19.835534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cad105-f297-4142-a479-89a41699066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:19.836630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cad105-f297-4142-a479-89a41699066d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:19.837112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cad105-f297-4142-a479-89a41699066d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:19.837596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cad105-f297-4142-a479-89a41699066d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:19.838447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cad105-f297-4142-a479-89a41699066d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:19.840080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cad105-f297-4142-a479-89a41699066d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-27cad105-f297-4142-a479-89a41699066d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:19.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:19.853491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:19.859839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 501/1010] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:19.862425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:19.895320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:19.961675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ee033-584f-43e6-9cbc-a3cd81bf6e59 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:19.997138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:19.998129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:19.998286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:19.998603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:19.999331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:19.999635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:20.001007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:20.010583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 510/1011] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:20.012034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cad105-f297-4142-a479-89a41699066d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:20.120475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cad105-f297-4142-a479-89a41699066d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:20.156107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:20.156794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:20.157071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:20.157347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:20.158076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:20.158381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:20.159812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5b7bf-5e52-451c-b7d2-2888c19c5527 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:20.165437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 502/1012] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Jun 24 10:04:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:20.171363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:20.269862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:20.274234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384333-4ab1-4c85-80e3-42316df653a9 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:20.309850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0efb5-e87b-4640-8489-fcf817af1e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:20.310661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0efb5-e87b-4640-8489-fcf817af1e1c None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:20.310974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0efb5-e87b-4640-8489-fcf817af1e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:20.311250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0efb5-e87b-4640-8489-fcf817af1e1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:20.312007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0efb5-e87b-4640-8489-fcf817af1e1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:20.313391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0efb5-e87b-4640-8489-fcf817af1e1c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:20.333101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 511/1013] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:04:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:20.985032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc69017-3309-4107-89b4-7407e6764915 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:20.985557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc69017-3309-4107-89b4-7407e6764915 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:20.985557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc69017-3309-4107-89b4-7407e6764915 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:20.986887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 503/1014] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:20.992378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:20.992793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:20.992961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:21.009867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:21.011078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:21.080741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6cff-cba0-4503-a831-625987bc4656 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:21.081026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6cff-cba0-4503-a831-625987bc4656 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:21.081305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6cff-cba0-4503-a831-625987bc4656 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:21.082871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 504/1015] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:21.089675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 512/1016] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:20 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:21.089832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:21.089832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:21.089832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:21.105923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:21.107219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:21.109064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:21.137193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:21.182189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 505/1017] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:21.194522 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:21.206269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:21.216835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b676b3e-3b04-41ff-ac1d-0e2bdb1082fc None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:21.254344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:21.254983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:21.255268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:21.255589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:21.256285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:21.257702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:21.274667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 513/1018] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:21 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:21.316099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361073e-0c5a-4916-aeef-ed5b6489f6f5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:21.364321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794b425-4dbf-43e7-aace-0d22ece42265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:21.365441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794b425-4dbf-43e7-aace-0d22ece42265 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:21.365889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794b425-4dbf-43e7-aace-0d22ece42265 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:21.366357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794b425-4dbf-43e7-aace-0d22ece42265 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:21.367844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794b425-4dbf-43e7-aace-0d22ece42265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:21.369549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794b425-4dbf-43e7-aace-0d22ece42265 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:21.376714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 506/1019] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:21.378274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:21.495353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7caa-0e60-4b1b-9400-9c4abb118cb8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:21.531238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201318ca-f174-4041-95aa-fabb2d569fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:21.531837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201318ca-f174-4041-95aa-fabb2d569fb1 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:21.532116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201318ca-f174-4041-95aa-fabb2d569fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:21.532390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201318ca-f174-4041-95aa-fabb2d569fb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:21.533123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201318ca-f174-4041-95aa-fabb2d569fb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:21.534608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201318ca-f174-4041-95aa-fabb2d569fb1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:21.546125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:21.570744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 514/1020] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:21 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:22.456322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b190f80-6ba9-4867-8857-412de487cb3b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:22.456878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b190f80-6ba9-4867-8857-412de487cb3b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:22.456878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b190f80-6ba9-4867-8857-412de487cb3b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:22.458234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 507/1021] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:22.463699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:22.464145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:22.464225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:22.481703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:22.483300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:22.553546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 515/1022] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:22.577010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b190f80-6ba9-4867-8857-412de487cb3b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:22.605843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:22.684891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b190f80-6ba9-4867-8857-412de487cb3b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:22.725299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:22.726138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:22.726409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:22.726682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:22.737473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:22.745766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:22.746730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:22.747749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 508/1023] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Jun 24 10:04:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:22.754850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:22.855587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba999bc8-f284-4720-a6e5-809b0b4e46fa None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:22.878073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:22.890902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:22.891588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:22.891871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:22.892148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:22.892900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:22.893214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:22.894599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:22.903300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 516/1024] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:04:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:22.909033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:22.996301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.012451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1c842-0593-4b6b-a67f-c1140bb9a954 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.046394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.047235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.047514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.047787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.059247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.060798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:23.077922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:23.079217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:23.080132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:23.080132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:23.080132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:23.080132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:23.083879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 509/1025] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:23.085249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.193173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7596-5cad-4d87-ae92-553af85eed91 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.232626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.233381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.233663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.233941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.234674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:23.235058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.236504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208ff6e-9971-43eb-a7cf-49123bc6f912 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.253037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 517/1026] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:23.254629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.333843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df279-6d88-4983-b886-9896ae09aa91 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.334089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df279-6d88-4983-b886-9896ae09aa91 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.334396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df279-6d88-4983-b886-9896ae09aa91 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.335969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 518/1027] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:23.340839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.341101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.341528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.359407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:23.360589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:23.363545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fae03-f257-4efe-9a52-8eba50ab4efa None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.399705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.400496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.400773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.401041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.402077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.409632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.411250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.413189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:23.426698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:23.427713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:23.428699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 510/1028] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:23.433579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.436945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 519/1029] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:23.459437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.541106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64381287-af80-40d7-87e9-94dba9927691 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.563687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996ed0b-9e20-45dd-823d-3dbf50446321 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.575117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.575872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.576473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.576473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.577247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:23.577554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.579037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.601340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 511/1030] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:23.602360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.602469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.603307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.603590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.603894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] PATH_INFO: `/v3/services/image` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.604931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.606324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:23.618949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:23.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:23.621431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 520/1031] 10.222.0.64 () {56 vars in 1105 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:23.626250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.664438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.667303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.711823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a0943-6493-48e6-bf06-f63222bfee29 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.729040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640618-5e26-4cd7-82d6-3eddc8614c35 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.746094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bbbfd-3c23-4f25-9092-1b8546c52886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.746782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bbbfd-3c23-4f25-9092-1b8546c52886 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.747070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bbbfd-3c23-4f25-9092-1b8546c52886 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.747349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bbbfd-3c23-4f25-9092-1b8546c52886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.762009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.762689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.763009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.763346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.764145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] RBAC: Adding query filter params (name=image) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:23.764455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.765867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46c00c-e9d4-4eed-9ec7-a2e29be088cb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.770719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 521/1032] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:23.776158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cdaf-4028-4436-ad58-c8f5ccaac8a4 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.776539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cdaf-4028-4436-ad58-c8f5ccaac8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.776812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cdaf-4028-4436-ad58-c8f5ccaac8a4 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.778087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 522/1033] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:23.778892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785cdaf-4028-4436-ad58-c8f5ccaac8a4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:23.814987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bbbfd-3c23-4f25-9092-1b8546c52886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.816359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bbbfd-3c23-4f25-9092-1b8546c52886 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.869516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.885724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785cdaf-4028-4436-ad58-c8f5ccaac8a4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:23.888413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.914552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 512/1034] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:23.916181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.916544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.916827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.927070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:23.927737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:23.928018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:23.928291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:23.928997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] RBAC: Adding query filter params (type=image) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:23.929302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:23.930674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:23.933764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:23.934990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:23.935555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 523/1035] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:23.936324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.002037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 513/1036] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:24.002925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.048630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046f83c-4835-4ec7-90de-e1c25d22fb8c None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.081310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.082039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.082320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.082596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.086603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:24.093480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.094973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.103163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0ecaf-d68c-43ef-98df-8b7ddabab42e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:24.112037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:24.113045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:24.113986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:24.113986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:24.113986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:24.113986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:24.113986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 524/1037] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:24.114280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.133318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:24.145125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.145782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.146061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.146336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.147161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] RBAC: Adding query filter params (interface=public, service_id=1c772486308143d3860f142dfec9c0b0, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:24.147471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.148850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.153773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 514/1038] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/endpoints?service_id=1c772486308143d3860f142dfec9c0b0&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:24.154605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.221148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9395aae-8c0b-4a3f-b517-4e9fc64614b9 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.255082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.255540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803101f-c3df-40ca-bd60-91aca2a8c5df None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.255890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.256172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.256448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.257667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.259158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.260782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 525/1039] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:24.261537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.290396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.291084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.291366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.291641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.292370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:24.292676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.294064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.305188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 515/1040] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:24.306159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.318226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:24.349795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:24.365409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f9e3-a5b6-42ff-b071-3efd903098b0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.399134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.399808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.400092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.400369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.401064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.402448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.407068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716ff4c-c974-4d73-9d50-5f36af06b654 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.409917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 526/1041] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:24.410445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.448733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.449459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.449726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.450011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.450788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.452551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.487434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 516/1042] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:04:24 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:24.536698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78f3ec-419e-4d79-b33b-0dfdb3b97307 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.580385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.581322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.581662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.581799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.590497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.592105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:24.608385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:24.609385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:24.610287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 527/1043] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:24.616042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.686268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:24.730061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f99bbe-2289-46f4-97c3-072ff174a676 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.763628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.764299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.764583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.764857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.765655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:24.765962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.767438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de425cf-454f-413b-b37b-6ae1bd0aa796 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.782734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 517/1044] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:24.786156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:24.798908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:24.894835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091ab0d-c217-4220-9145-5607554b35fe None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:24.930924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:24.931590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:24.931872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:24.932145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:24.932978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] RBAC: Adding query filter params (role.id=e2c83d048daa420bb9f1204899da2381, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ac88daa9b11b4e13b89f3f981686ac72) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:24.933290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:24.934772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaab61-761c-4f2a-aec1-4c5c7083baa3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:24.946750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 528/1045] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:24 2022] GET /identity/v3/role_assignments?user.id=ac88daa9b11b4e13b89f3f981686ac72&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=e2c83d048daa420bb9f1204899da2381 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:26.510302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782beb6b-f413-4601-95c5-b77fc752cf86 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.510916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782beb6b-f413-4601-95c5-b77fc752cf86 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.510916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782beb6b-f413-4601-95c5-b77fc752cf86 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.512297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 518/1046] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:26.518277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.518856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.518856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.536345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:26.537525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:26.597719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b9f23-c634-4987-bb6a-8b131fdb931b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.597980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b9f23-c634-4987-bb6a-8b131fdb931b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.598265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b9f23-c634-4987-bb6a-8b131fdb931b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.599931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 519/1047] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:26.605207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.605467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.605742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.613693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 529/1048] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:26.624686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:26.625910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:26.637310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.640476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:26.706051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 520/1049] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:26.728029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.752072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91794d11-9719-4d5c-9321-f43158763f6b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.787675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:26.788396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.788673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.788955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] PATH_INFO: `/v3/services/image` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.789614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:26.789876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:26.791024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:26.803210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:26.804246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:26.805187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 530/1050] 10.222.0.64 () {56 vars in 1105 bytes} [Fri Jun 24 10:04:26 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:26.810409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.844359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea6c7c-2c12-4ca8-941a-a20349cc3af3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.882324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:26.882548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:26.883578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.884147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.884147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.895425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:26.903049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:26.904056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:26.905016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 521/1051] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:26.910616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.922109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e98b-be4c-47eb-839f-6c79c88aa3af None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.960158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:26.961084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.961453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.961839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.962795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] RBAC: Adding query filter params (name=image) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:26.963282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:26.964967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9d314-458c-477c-b721-2542f603c67e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:26.975042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 531/1052] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:26.975042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7c805-46e2-401f-915a-15d1b6aa01e8 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.975042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7c805-46e2-401f-915a-15d1b6aa01e8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.975042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7c805-46e2-401f-915a-15d1b6aa01e8 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.975042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 532/1053] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:26.975887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7c805-46e2-401f-915a-15d1b6aa01e8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.024691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebfd4c-5f1a-4c1c-bcb6-7784d791f2b8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.040573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.059987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.060796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.061078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.061352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.062088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:27.062396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.063834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc6f2-ed17-498e-a10e-6cf949113588 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.072075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 522/1054] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:27.073074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.073364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.073646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.081531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7c805-46e2-401f-915a-15d1b6aa01e8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.089470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:27.090672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:27.110185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.115589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.116299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.116582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.116862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.117574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] RBAC: Adding query filter params (type=image) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:27.117883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.119306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.129208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 533/1055] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:27.129208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.161422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 523/1056] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:27.162241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.232361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753724-f372-460b-ab84-d11bc860ae7a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.262894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc95b9-4235-46e3-8e6e-17327ad7710d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.265793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.266674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.266987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.267263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] PATH_INFO: `/v3/users/3a764694afc84290ad13a603fbb272bd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.272339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.280832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] RBAC: Authorizing `identity:get_user(user_id=3a764694afc84290ad13a603fbb272bd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.282300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.284048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 534/1057] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/users/3a764694afc84290ad13a603fbb272bd => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:27.284834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.300770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.301369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.301648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.301918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.302667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.304079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.323551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 524/1058] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:27.324188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.344231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.402601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759bef-b9d2-4973-b7be-eb4b0c3f3649 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.448836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.449681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.449976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.450250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.458550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d174181-3be5-4ea1-9482-925cd2f01c35 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.461980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:27.470698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:27.471789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:27.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 535/1059] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:27.478448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.502213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.502938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.503504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.503504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.543124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.579020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.580402 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.603058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe71b58-8bf5-49a7-872b-7249dbdc321a None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.620452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.647545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.648235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.648521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.648799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.649558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:27.649869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.651289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.659729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 536/1060] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:27.665375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.692325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 525/1061] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:27.697916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.768449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.773324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9855fe7-01b5-46bd-9399-c71c98a5c1e4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.808232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4790e24-72cd-4c56-a1dd-fe45586327f7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.810567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.811647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.811647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.811944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.825200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.825377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.843598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:27.844272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:27.845151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:27.846070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:27.846070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:27.846070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:27.846070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:27.846300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.846300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.846300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.846518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 537/1062] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:27.847586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=8cee161de5684de6ab8596790bc803bc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.849047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.850819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 526/1063] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/projects/8cee161de5684de6ab8596790bc803bc => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:27.851344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.857345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.880670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.961677 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4d549-1668-4dde-bae6-71b9119177fd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.962321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404ad62-3777-4af4-a12b-2d0736041ff7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.993882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.997972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.998657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.998972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.999249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.999452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.999983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:28.000291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.000475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:28.000745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:28.001368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:28.001745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.002231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=8cee161de5684de6ab8596790bc803bc, user.id=3a764694afc84290ad13a603fbb272bd) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:28.002544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.004012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.016201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 527/1064] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/role_assignments?user.id=3a764694afc84290ad13a603fbb272bd&scope.project.id=8cee161de5684de6ab8596790bc803bc&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:28.026797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 538/1065] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:28.032908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:28.146447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b5892-8000-4563-8b4b-bedd0d1bade5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:28.185296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:28.186108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:28.186391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:28.186665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:28.194635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.196124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:28.210171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:28.211186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:28.212158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 528/1066] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:04:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:28.217383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:28.306822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:28.325521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3e6c5-737d-4bd9-af27-803de2f19a78 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:28.362232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:28.363203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:28.363634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:28.364058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:28.365107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:28.365581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.367223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.390800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 539/1067] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:04:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:28.392036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:28.452264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:28.500255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeb98a-f78c-4517-a6f5-3a622f686d8b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:28.536212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:28.537028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:28.537280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:28.537547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:28.549189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.550647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:28.568689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:28.569625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:28.570517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:28.570517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:28.570517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:28.570517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:28.570517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 529/1068] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:28.575670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:28.682119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a91ba-8f4f-4a8f-af55-19782e17b12e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:28.722377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:28.723102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:28.723385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:28.723663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:28.724393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:28.724702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.726091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.738210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 540/1069] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:28.743714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:28.847842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f730ac-0495-43a7-8683-753f1f99986e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:28.858572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:28.884133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:28.884934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:28.885215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:28.885486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:28.892320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:28.893777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:28.908007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:28.908880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:28.914883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 530/1070] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:28.915491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:28.918221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:29.018328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e7820-93c0-462e-9ea2-2c76b3abe2c0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:29.053295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:29.053969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:29.054251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:29.054537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:29.055369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:29.055668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:29.057148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312170b2-80fb-44d4-8780-015d6410f94b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:29.070955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 541/1071] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:29.076734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:29.180350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc396c-adbe-4cea-afd5-dcd32e67236f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:29.214511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5dc2e-6f8e-473f-9145-6580f0711bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:29.215437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5dc2e-6f8e-473f-9145-6580f0711bda None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:29.215720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5dc2e-6f8e-473f-9145-6580f0711bda None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:29.215995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5dc2e-6f8e-473f-9145-6580f0711bda None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ac88daa9b11b4e13b89f3f981686ac72/roles/e2c83d048daa420bb9f1204899da2381` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:29.218311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5dc2e-6f8e-473f-9145-6580f0711bda None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=ac88daa9b11b4e13b89f3f981686ac72, role_id=e2c83d048daa420bb9f1204899da2381)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:29.219786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5dc2e-6f8e-473f-9145-6580f0711bda None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:29.233573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 531/1072] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:04:29 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/ac88daa9b11b4e13b89f3f981686ac72/roles/e2c83d048daa420bb9f1204899da2381 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:29.432220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc60d11-8a6f-4582-ae6a-50810bff648a None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:29.432719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc60d11-8a6f-4582-ae6a-50810bff648a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:29.433000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc60d11-8a6f-4582-ae6a-50810bff648a None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:29.434540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 542/1073] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:29.440468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:29.440729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:29.441023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:29.458916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:29.460101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:29.485945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:29.534287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 532/1074] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:29.557486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc60d11-8a6f-4582-ae6a-50810bff648a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:29.627035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:29.671170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc60d11-8a6f-4582-ae6a-50810bff648a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:29.720892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd3293-0aca-4d07-91cf-a8f996aae8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:29.721610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3293-0aca-4d07-91cf-a8f996aae8ea None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:29.721837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3293-0aca-4d07-91cf-a8f996aae8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:29.722242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3293-0aca-4d07-91cf-a8f996aae8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:29.793018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd3293-0aca-4d07-91cf-a8f996aae8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:29.794603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd3293-0aca-4d07-91cf-a8f996aae8ea None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:29.855480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:29.901766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 543/1075] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:29.903235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.017049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f13d9-9242-4f41-8b2c-410026785144 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.033337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7799-0a82-4073-8e1f-beed94563f7d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.033654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7799-0a82-4073-8e1f-beed94563f7d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.033951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7799-0a82-4073-8e1f-beed94563f7d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.035517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 544/1076] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:04:30.040355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.040619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.040926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.053281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.054077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.054355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.054625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.057627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:30.058937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:30.064939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:30.066395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:30.084066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:30.085117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:30.086053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:30.086053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:30.086053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:30.086053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:30.086053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 533/1077] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:30.091178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.105256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:30.140647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 545/1078] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:30.163421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.199847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fb762-b6aa-4a36-afe0-8647cb3c8eb9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.236789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.237463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.237747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.238023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.238789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:30.239105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:30.240501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:30.252043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 534/1079] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:30.261803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:30.273523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce637b8-226d-463a-8b85-56b4d03784f5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.315188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.316024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.316291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.316576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.330233 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:30.338294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:30.339415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:30.340280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 546/1080] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:30.346301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.426841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:30.460828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af7582-f572-4212-aca9-7a8e78a298e3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.499585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.500265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.500530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.500807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.501567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:30.501875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:30.503465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:30.512499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 535/1081] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:30.518577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.624513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43657a5-55cf-489b-9095-17948ffcfb62 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.666202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.667291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.667572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.667849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] PATH_INFO: `/v3/users/3a764694afc84290ad13a603fbb272bd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.669140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] RBAC: Authorizing `identity:get_user(user_id=3a764694afc84290ad13a603fbb272bd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:30.670739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:30.672384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 547/1082] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:30 2022] GET /identity/v3/users/3a764694afc84290ad13a603fbb272bd => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:30.678225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.779216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94215d51-964b-457f-911e-4277fa6ca7c4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.808721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:30.814459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c10eea-a574-48a3-9b00-f95354438f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.815359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10eea-a574-48a3-9b00-f95354438f43 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.815641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10eea-a574-48a3-9b00-f95354438f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.815919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10eea-a574-48a3-9b00-f95354438f43 None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.817115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c10eea-a574-48a3-9b00-f95354438f43 None admin] RBAC: Authorizing `identity:get_project(project_id=8cee161de5684de6ab8596790bc803bc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:30.818566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c10eea-a574-48a3-9b00-f95354438f43 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:30.820340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 536/1083] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:30 2022] GET /identity/v3/projects/8cee161de5684de6ab8596790bc803bc => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:30.826363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:30.849947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:30.929858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410e0b3-3e66-4d6c-935b-f0ebfbda63cc None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:30.964386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8435-a690-4b20-b859-0345913c6730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:30.965282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8435-a690-4b20-b859-0345913c6730 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:30.965560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8435-a690-4b20-b859-0345913c6730 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:30.965837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8435-a690-4b20-b859-0345913c6730 None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc/users/3a764694afc84290ad13a603fbb272bd/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:30.967959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8435-a690-4b20-b859-0345913c6730 None admin] RBAC: Authorizing `identity:create_grant(project_id=8cee161de5684de6ab8596790bc803bc, user_id=3a764694afc84290ad13a603fbb272bd, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:30.969418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8435-a690-4b20-b859-0345913c6730 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:30.983681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 548/1084] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:04:30 2022] PUT /identity/v3/projects/8cee161de5684de6ab8596790bc803bc/users/3a764694afc84290ad13a603fbb272bd/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:04:31.256365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e28be-773a-4804-ad9a-9ca48379bbf4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:31.256881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e28be-773a-4804-ad9a-9ca48379bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:31.256881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e28be-773a-4804-ad9a-9ca48379bbf4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:31.258311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 537/1085] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:31.263641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:31.263933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:31.264249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:31.281564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:31.282766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:31.355129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 549/1086] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:31.379080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e28be-773a-4804-ad9a-9ca48379bbf4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:31.484040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e28be-773a-4804-ad9a-9ca48379bbf4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:31.518579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:31.519338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:31.519620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:31.519892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:31.530275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:31.538487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:31.539473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:31.540296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 538/1087] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Jun 24 10:04:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:31.546296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:31.558347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:31.649587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1668a2-6dc4-4632-8542-5c3a586b2cc7 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:31.685418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:31.686260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:31.686633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:31.686983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:31.687683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:31.687988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:31.689375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:31.701034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 550/1088] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:04:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:31.706857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:31.768916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:31.811470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f0808-0b9e-4ff3-a616-4c542c438ecc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:31.846122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:31.846942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:31.847222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:31.847494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:31.858750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:31.860240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:31.877433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:31.878320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:31.879301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:31.879301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:31.879301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:31.879301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:31.879301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 539/1089] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:31.884668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:31.959982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:31.992848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab996aa9-3963-4fa2-9550-b457cc43c8f4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:32.027939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:32.028535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:32.028818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:32.029087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:32.029812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:32.030119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:32.031531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:32.043773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 551/1090] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:32.049521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:32.157478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7d405-9865-425d-a284-c71ec1463a84 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:32.170604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:32.193481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:32.194278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:32.194561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:32.194871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:32.203155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:32.204619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:04:32.218477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:32.219496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:32.220979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 540/1091] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Jun 24 10:04:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:32.225690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:32.328556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f46b7-7d27-424e-8a00-91933057ad97 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:32.358128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:32.364514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:32.365193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:32.365473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:32.365747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:32.366587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:32.366920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:32.368359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:32.385481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 552/1092] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Jun 24 10:04:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:32.391601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:32.499441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72a18-f882-4cf7-8578-2e1a130e76ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:32.531823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:32.532463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:32.532744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:32.533022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:32.573544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:32.601347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:32.602787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:32.704531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 541/1093] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:32.710385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:32.827997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee4889-cf3e-4789-93cb-b1c74ee7708f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:32.864297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:32.865028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:32.865307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:32.865582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:32.875696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:32.877153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-6057f4e3-70d1-49ce-b2b0-a416fd8763fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:04:32.894680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:32.896008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:32.897387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:04:32.897387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:04:32.897387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:04:32.897387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:32.897387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 553/1094] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:32.901777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:32.973244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:32.993406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323ca4-be4c-4338-afbd-6f8aa3107281 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:32.993641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323ca4-be4c-4338-afbd-6f8aa3107281 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:32.993938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323ca4-be4c-4338-afbd-6f8aa3107281 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:32.995334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 554/1095] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:33.000481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.000771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.001048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.004817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1fd09-6624-4d85-a37e-0b55536a9060 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.017808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:33.019066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:33.038930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.039593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.039878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.040153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.040866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:33.041169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.042558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.055243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 542/1096] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:33.061845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.063753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:33.099088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 555/1097] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:04:33.123254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.163716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d92cd-920a-47cb-bb06-8eee65399fb3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.192876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:33.198253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.199078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.199361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.199646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.206920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.208390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.222138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c5a67-49a5-430c-89ff-fa18bc97aae1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:04:33.222356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:33.223868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:33.224759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 543/1098] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:33.230275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.259631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.260430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.260716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.261083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.271414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:33.279011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:33.279978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:33.281001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 556/1099] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:04:33.286186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.319732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:33.334480 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f49375-6e46-44fe-be36-788250db95c6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.367877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.368538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.368817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.369087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.369879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:33.370186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.371676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.384484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 544/1100] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:33.391811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.392395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7982c-c87b-4adb-82e0-9d8a5552cf44 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.427128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.427809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.428091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.428366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.429124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:33.429433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.430849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.439566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 557/1101] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:33.443797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:33.445728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.493070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9f899-8ce8-4a1c-82c4-26cbc86456bb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.527045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.527699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.527981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.528259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.529297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] RBAC: Adding query filter params (role.id=e2c83d048daa420bb9f1204899da2381, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=ac88daa9b11b4e13b89f3f981686ac72) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:33.529747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.531235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.537023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 545/1102] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/role_assignments?user.id=ac88daa9b11b4e13b89f3f981686ac72&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=e2c83d048daa420bb9f1204899da2381 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:04:33.543585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:33.551932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abaea4-2fc9-4a6d-9368-51cc1569ec4b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.590172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.591106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.591425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.591697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] PATH_INFO: `/v3/users/3a764694afc84290ad13a603fbb272bd` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.593174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=3a764694afc84290ad13a603fbb272bd)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.594678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.596478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 558/1103] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/users/3a764694afc84290ad13a603fbb272bd => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:33.602594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:33.714620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1174c9-8052-4b26-9aaa-eba8270e8520 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:33.731645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:33.751219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:33.751899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:33.752173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:33.752446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:33.822373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:33.823883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:33.924210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 546/1104] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:33.930391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:34.036254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51635110-d0e8-4163-b1b5-474525f71ee4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:34.071282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b08f3-1dc3-4789-bc72-3624a35168d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:34.072155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b08f3-1dc3-4789-bc72-3624a35168d9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:34.072430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b08f3-1dc3-4789-bc72-3624a35168d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:34.072705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b08f3-1dc3-4789-bc72-3624a35168d9 None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:34.073908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b08f3-1dc3-4789-bc72-3624a35168d9 None admin] RBAC: Authorizing `identity:get_project(project_id=8cee161de5684de6ab8596790bc803bc)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:34.075411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b08f3-1dc3-4789-bc72-3624a35168d9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:34.077156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 559/1105] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:33 2022] GET /identity/v3/projects/8cee161de5684de6ab8596790bc803bc => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:34.083469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:34.102721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:34.185052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cda10-b831-4e7a-9fc0-24633e631dbc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:34.218872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:34.219524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:34.219823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:34.220081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:34.220915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=8cee161de5684de6ab8596790bc803bc, user.id=3a764694afc84290ad13a603fbb272bd) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:34.221270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:34.222661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee1308-c0a7-44e4-bffd-754b9bca61c8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:34.228984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 547/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:34 2022] GET /identity/v3/role_assignments?user.id=3a764694afc84290ad13a603fbb272bd&scope.project.id=8cee161de5684de6ab8596790bc803bc&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:36.305893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fc8c9-6b0f-4d92-b418-b873c0369c22 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:36.306867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fc8c9-6b0f-4d92-b418-b873c0369c22 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:36.306867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fc8c9-6b0f-4d92-b418-b873c0369c22 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:36.308997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 560/1107] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:36.314615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:36.315071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:36.315182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:36.332481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:36.333670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:36.408176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 548/1108] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:36.433582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fc8c9-6b0f-4d92-b418-b873c0369c22 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:36.467116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:36.542934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fc8c9-6b0f-4d92-b418-b873c0369c22 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:36.588034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c9b28-e5c3-42ff-908a-8539e33452b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:36.588768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9b28-e5c3-42ff-908a-8539e33452b9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:36.589051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9b28-e5c3-42ff-908a-8539e33452b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:36.589325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9b28-e5c3-42ff-908a-8539e33452b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:36.590575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9b28-e5c3-42ff-908a-8539e33452b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:36.592661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9b28-e5c3-42ff-908a-8539e33452b9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:36.594331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 561/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:36.600363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:36.613059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:36.705943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd8a1f-2a4d-46c8-9678-c3a53f57dd5b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:36.741700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204daa-962d-4ac9-ab86-47e65b5de09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:36.742356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204daa-962d-4ac9-ab86-47e65b5de09b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:36.742634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204daa-962d-4ac9-ab86-47e65b5de09b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:36.742943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204daa-962d-4ac9-ab86-47e65b5de09b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:36.743653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204daa-962d-4ac9-ab86-47e65b5de09b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:36.745107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204daa-962d-4ac9-ab86-47e65b5de09b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:36.768545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 549/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:04:36 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:38.836956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eac7d-ef79-4fba-b299-e7ec5e9b6e37 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:38.837893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eac7d-ef79-4fba-b299-e7ec5e9b6e37 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:38.837893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eac7d-ef79-4fba-b299-e7ec5e9b6e37 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:38.840110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 562/1111] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:38.845803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:38.846260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:38.846359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:38.866969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:38.868167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:38.933592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:38.945208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 550/1112] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:38.972603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eac7d-ef79-4fba-b299-e7ec5e9b6e37 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:39.103562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:39.111175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eac7d-ef79-4fba-b299-e7ec5e9b6e37 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:39.156381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:39.157484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:39.157804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:39.158082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] PATH_INFO: `/v3/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:39.160227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] RBAC: Authorizing `identity:get_role(role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:39.161628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:39.163100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 563/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:38 2022] GET /identity/v3/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 223 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:39.169592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:39.289420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30934af-0043-4651-b5a2-fd07882add9f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:39.332461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:39.333334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:39.333613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:39.333890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] PATH_INFO: `/v3/users/1cba35cdf2654f67bc93f5f69a35fabe` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:39.352093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] RBAC: Authorizing `identity:get_user(user_id=1cba35cdf2654f67bc93f5f69a35fabe)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:39.353749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:39.355907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 551/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:39 2022] GET /identity/v3/users/1cba35cdf2654f67bc93f5f69a35fabe => generated 286 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:39.362859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:39.491479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8c1ce-604a-47de-be35-a9fe5add0c62 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:39.531252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:39.531916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:39.532195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:39.532467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:39.576393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:39.616043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:39.617518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:39.733385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 564/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:39.739665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:39.798212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:39.846931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5baa-7755-47c9-a6da-76a87a638b25 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:39.883747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f38835-e25f-4373-ac13-63cc12f47e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:39.884601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38835-e25f-4373-ac13-63cc12f47e20 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:39.884876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38835-e25f-4373-ac13-63cc12f47e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:39.885143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38835-e25f-4373-ac13-63cc12f47e20 None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:39.886321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f38835-e25f-4373-ac13-63cc12f47e20 None admin] RBAC: Authorizing `identity:get_project(project_id=8cee161de5684de6ab8596790bc803bc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:39.887796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f38835-e25f-4373-ac13-63cc12f47e20 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:39.889550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 552/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:39 2022] GET /identity/v3/projects/8cee161de5684de6ab8596790bc803bc => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:39.896073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:39.980326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:40.019183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6d264-6503-4960-a475-31dfc93669b6 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:40.056563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:40.057223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:40.057503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:40.057776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:40.058617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] RBAC: Adding query filter params (role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=8cee161de5684de6ab8596790bc803bc, user.id=1cba35cdf2654f67bc93f5f69a35fabe) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:40.058962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:40.060405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55121d-f742-49ae-a651-215de996cd71 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:40.069268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 565/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:39 2022] GET /identity/v3/role_assignments?user.id=1cba35cdf2654f67bc93f5f69a35fabe&scope.project.id=8cee161de5684de6ab8596790bc803bc&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:42.051727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bb20-536f-440b-af65-d0473feabaed None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:42.052283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bb20-536f-440b-af65-d0473feabaed None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:42.052283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bb20-536f-440b-af65-d0473feabaed None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:42.053570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 553/1118] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:42.059995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:42.060730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:42.060924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:42.080565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:42.081765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:42.151723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 566/1119] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:42.175609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bb20-536f-440b-af65-d0473feabaed None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:42.285495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bb20-536f-440b-af65-d0473feabaed None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:42.308704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:42.320706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:42.321561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:42.321833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:42.322098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] PATH_INFO: `/v3/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:42.323721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] RBAC: Authorizing `identity:get_role(role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:42.325095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:42.326431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 554/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:42 2022] GET /identity/v3/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:42.333115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:42.395422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:42.442586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711fec-dfdd-401a-8d20-8a542df99c3d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:42.481978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:42.482912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:42.483192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:42.483463 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] PATH_INFO: `/v3/users/1cba35cdf2654f67bc93f5f69a35fabe` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:42.484978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] RBAC: Authorizing `identity:get_user(user_id=1cba35cdf2654f67bc93f5f69a35fabe)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:42.486517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:42.488201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 567/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:42 2022] GET /identity/v3/users/1cba35cdf2654f67bc93f5f69a35fabe => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:42.497578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:42.607869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcc075-987e-44b4-aa2d-b5b34d3c3c7c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:42.644504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e7563-52b2-4290-b4a5-e07cb4f7c713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:42.645360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e7563-52b2-4290-b4a5-e07cb4f7c713 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:42.645640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e7563-52b2-4290-b4a5-e07cb4f7c713 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:42.645910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e7563-52b2-4290-b4a5-e07cb4f7c713 None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:42.647121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e7563-52b2-4290-b4a5-e07cb4f7c713 None admin] RBAC: Authorizing `identity:get_project(project_id=8cee161de5684de6ab8596790bc803bc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:42.648564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e7563-52b2-4290-b4a5-e07cb4f7c713 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:42.650241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 555/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:42 2022] GET /identity/v3/projects/8cee161de5684de6ab8596790bc803bc => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:42.660513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:42.777304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534989-0f03-4f80-ab4b-f1ae7fcc22ec None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:42.804767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:42.814723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348469df-5ee9-4631-847b-cf093822956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:42.815634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348469df-5ee9-4631-847b-cf093822956c None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:42.815919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348469df-5ee9-4631-847b-cf093822956c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:42.816213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348469df-5ee9-4631-847b-cf093822956c None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc/users/1cba35cdf2654f67bc93f5f69a35fabe/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:42.818613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348469df-5ee9-4631-847b-cf093822956c None admin] RBAC: Authorizing `identity:create_grant(project_id=8cee161de5684de6ab8596790bc803bc, user_id=1cba35cdf2654f67bc93f5f69a35fabe, role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:42.820104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348469df-5ee9-4631-847b-cf093822956c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:42.838286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 568/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:04:42 2022] PUT /identity/v3/projects/8cee161de5684de6ab8596790bc803bc/users/1cba35cdf2654f67bc93f5f69a35fabe/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:44.813721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473fc9e-3e55-4343-ac5f-a61631417276 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:44.814607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473fc9e-3e55-4343-ac5f-a61631417276 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:44.814607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473fc9e-3e55-4343-ac5f-a61631417276 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:44.817223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 556/1124] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:44.825937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:44.826372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:44.826513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:44.845597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:44.846830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:44.920599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 569/1125] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:44.944728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473fc9e-3e55-4343-ac5f-a61631417276 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:45.020918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:45.065825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473fc9e-3e55-4343-ac5f-a61631417276 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:45.100979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:45.101827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:45.102105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:45.102379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] PATH_INFO: `/v3/roles/9fd9a22adc114fb4abf9afe0d4dd056c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:45.103925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] RBAC: Authorizing `identity:get_role(role_id=9fd9a22adc114fb4abf9afe0d4dd056c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:45.105678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:45.107083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 557/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:44 2022] GET /identity/v3/roles/9fd9a22adc114fb4abf9afe0d4dd056c => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:45.113275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:45.218338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6514b9-9e2e-4475-ae75-00e32661d1c6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:45.236300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:45.255232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:45.256101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:45.256378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:45.256650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] PATH_INFO: `/v3/users/1cba35cdf2654f67bc93f5f69a35fabe` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:45.257894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] RBAC: Authorizing `identity:get_user(user_id=1cba35cdf2654f67bc93f5f69a35fabe)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:45.259382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:45.261001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 570/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:45 2022] GET /identity/v3/users/1cba35cdf2654f67bc93f5f69a35fabe => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:45.267178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:45.371465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b68e21-b1be-4eae-9cd3-5f07d979c166 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:45.397271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:45.407623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:45.408263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:45.408539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:45.408814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:45.478188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:45.480044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:45.598745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 558/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 10:04:45.605550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:45.707393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b06ef-6ff8-485f-838f-7de8f1b6bf3d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:45.743875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d10c-b266-4491-ac0b-41f0a655b1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:45.744742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d10c-b266-4491-ac0b-41f0a655b1ed None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:45.745025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d10c-b266-4491-ac0b-41f0a655b1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:45.745304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d10c-b266-4491-ac0b-41f0a655b1ed None admin] PATH_INFO: `/v3/projects/8cee161de5684de6ab8596790bc803bc` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:45.746547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088d10c-b266-4491-ac0b-41f0a655b1ed None admin] RBAC: Authorizing `identity:get_project(project_id=8cee161de5684de6ab8596790bc803bc)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:45.748037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088d10c-b266-4491-ac0b-41f0a655b1ed None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:45.749750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 571/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:45 2022] GET /identity/v3/projects/8cee161de5684de6ab8596790bc803bc => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:45.756348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:45.784979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:45.868062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb773b6-8daf-4bc8-86d4-a06e3d95f6ae None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:45.904230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:45.904893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:45.905154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:45.905424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:45.906302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] RBAC: Adding query filter params (role.id=9fd9a22adc114fb4abf9afe0d4dd056c, scope.project.id=8cee161de5684de6ab8596790bc803bc, user.id=1cba35cdf2654f67bc93f5f69a35fabe) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:45.907151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:45.908602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8be6f4-9849-4e2a-8743-2c6874c900db None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:45.914592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 559/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:45 2022] GET /identity/v3/role_assignments?user.id=1cba35cdf2654f67bc93f5f69a35fabe&scope.project.id=8cee161de5684de6ab8596790bc803bc&role.id=9fd9a22adc114fb4abf9afe0d4dd056c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:47.952668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e05602-7d43-4359-809e-f08035b03d05 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:47.953621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e05602-7d43-4359-809e-f08035b03d05 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:47.953864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e05602-7d43-4359-809e-f08035b03d05 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:47.956199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 572/1131] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:47.962454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:47.962904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:47.962996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:47.984159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:47.985360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:48.056213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 560/1132] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:48.082590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e05602-7d43-4359-809e-f08035b03d05 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:48.143666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:48.195997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e05602-7d43-4359-809e-f08035b03d05 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:48.232752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e510-bc98-438a-bd6b-55da8610c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:48.235357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e510-bc98-438a-bd6b-55da8610c4aa None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:48.235357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e510-bc98-438a-bd6b-55da8610c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:48.235357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e510-bc98-438a-bd6b-55da8610c4aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:48.235357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1e510-bc98-438a-bd6b-55da8610c4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:48.237663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1e510-bc98-438a-bd6b-55da8610c4aa None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:48.238269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 573/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:48.244417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:48.266380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:48.356345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84e2f-d0ba-45f3-a6c8-e332b4445986 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:48.391406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ed244-e914-4de9-9df3-85b6de9776b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:48.392054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ed244-e914-4de9-9df3-85b6de9776b2 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:48.392328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ed244-e914-4de9-9df3-85b6de9776b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:48.392599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ed244-e914-4de9-9df3-85b6de9776b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:48.393315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ed244-e914-4de9-9df3-85b6de9776b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:48.394797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ed244-e914-4de9-9df3-85b6de9776b2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:48.420802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 561/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:04:48 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:50.492602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afe5cb-4c5a-4b08-9dfe-454f0bb634ab None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:50.493214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afe5cb-4c5a-4b08-9dfe-454f0bb634ab None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:50.493214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afe5cb-4c5a-4b08-9dfe-454f0bb634ab None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:50.494518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 574/1135] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:50.502511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:50.503253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:50.503390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:50.532695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:50.533922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:50.614525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 562/1136] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:50.650049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afe5cb-4c5a-4b08-9dfe-454f0bb634ab None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:50.754956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afe5cb-4c5a-4b08-9dfe-454f0bb634ab None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:50.777590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:50.797531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56e10f-ab99-4883-ba34-60653163b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:50.798333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56e10f-ab99-4883-ba34-60653163b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:50.798616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56e10f-ab99-4883-ba34-60653163b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:50.798924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56e10f-ab99-4883-ba34-60653163b6e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:50.800243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56e10f-ab99-4883-ba34-60653163b6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:50.801735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56e10f-ab99-4883-ba34-60653163b6e1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:50.803460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 575/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:04:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:50.809406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:50.811655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:50.922523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526fab5-7d3e-47d7-84f2-4f28349ccef2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:50.960167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4fcb8-4bae-437c-ac3f-8fe367acb134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:50.960829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4fcb8-4bae-437c-ac3f-8fe367acb134 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:50.961106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4fcb8-4bae-437c-ac3f-8fe367acb134 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:50.961380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4fcb8-4bae-437c-ac3f-8fe367acb134 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:50.962092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4fcb8-4bae-437c-ac3f-8fe367acb134 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:50.963571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4fcb8-4bae-437c-ac3f-8fe367acb134 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:50.986730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 563/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:04:50 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:53.055996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eba0-daac-49ef-a9e5-ebd6de4cb3d7 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.056824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eba0-daac-49ef-a9e5-ebd6de4cb3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.056824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8eba0-daac-49ef-a9e5-ebd6de4cb3d7 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.058974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 576/1139] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:53.067807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.068251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.068382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.091136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:53.092346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.150677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:53.167974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 564/1140] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:53.193384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8eba0-daac-49ef-a9e5-ebd6de4cb3d7 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.310021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8eba0-daac-49ef-a9e5-ebd6de4cb3d7 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.347951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.348744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.349029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.349303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.361373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:53.368960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:53.370062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:53.371204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 577/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:53.377841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.483440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a7ee0-b157-4c6c-8e2e-e819b540de35 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.528899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.529552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.529832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.530102 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.530874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:53.531253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.532715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.541051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 565/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:53.546984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.603677 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:53.659940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21034660-3e75-43c8-8f99-91d34eb8865d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.694369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.695280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.695563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.695832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] PATH_INFO: `/v3/users/b779fda5e5204e1987e6f9338e0eba0e` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.711290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] RBAC: Authorizing `identity:get_user(user_id=b779fda5e5204e1987e6f9338e0eba0e)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.712762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.714461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 578/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/users/b779fda5e5204e1987e6f9338e0eba0e => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:53.721052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.776291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:53.830433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274518-bd86-4178-977e-cabf6c0ce5c8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.866990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.867558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.867839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.868115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.946877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.948338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.997445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:54.051057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 566/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:04:54.057387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:54.178313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa273f36-e397-4e1e-8372-b9ce3ea2e5d4 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:54.187768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:54.214793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66606c-8868-40a3-afd9-c1767f4332d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:54.215656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66606c-8868-40a3-afd9-c1767f4332d3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:54.215938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66606c-8868-40a3-afd9-c1767f4332d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:54.216253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66606c-8868-40a3-afd9-c1767f4332d3 None admin] PATH_INFO: `/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:54.217545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66606c-8868-40a3-afd9-c1767f4332d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d423b6b2f1e9491d90a6fd2ea3357c53)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:54.219063 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66606c-8868-40a3-afd9-c1767f4332d3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:54.220825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 579/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:54 2022] GET /identity/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:54.231074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:54.349331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08b6cd-1a83-467e-a5da-e7d5e374c4c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:54.384479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:54.385157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:54.385431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:54.385707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:54.386537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=d423b6b2f1e9491d90a6fd2ea3357c53, user.id=b779fda5e5204e1987e6f9338e0eba0e) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:54.386879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:54.388312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f6f69-cd9d-4844-b817-1a6758bb7e82 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:54.394560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 567/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:04:54 2022] GET /identity/v3/role_assignments?user.id=b779fda5e5204e1987e6f9338e0eba0e&scope.project.id=d423b6b2f1e9491d90a6fd2ea3357c53&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:56.417501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20165-f9ae-453f-bd96-fabf7d25a8d8 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:56.418060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20165-f9ae-453f-bd96-fabf7d25a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:56.418060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20165-f9ae-453f-bd96-fabf7d25a8d8 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:56.420230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 580/1147] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:56.425733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:56.426181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:56.426270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:56.445169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:56.446355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:56.456059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:56.526696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 568/1148] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:56.550204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20165-f9ae-453f-bd96-fabf7d25a8d8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:56.659914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20165-f9ae-453f-bd96-fabf7d25a8d8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:56.696450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:56.697171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:56.697450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:56.697730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:56.709060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:56.716578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:56.717640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:04:56.718492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 581/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:56.724098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:56.826663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48605-9744-4ff9-ac2c-586da3561751 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:56.864311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:56.864969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:56.865247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:56.865524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:56.866255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:56.866556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:56.867971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:56.876224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 569/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:56.884913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:56.921384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:56.988451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efd007-4752-4025-b88d-0d89620079e8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:57.022944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:57.023809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:57.024093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:57.024364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] PATH_INFO: `/v3/users/b779fda5e5204e1987e6f9338e0eba0e` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:57.025618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] RBAC: Authorizing `identity:get_user(user_id=b779fda5e5204e1987e6f9338e0eba0e)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:57.027094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:57.028722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 582/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:56 2022] GET /identity/v3/users/b779fda5e5204e1987e6f9338e0eba0e => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:57.035090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:57.045467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:57.148940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a3319-a0b5-4211-896b-086e5b8bb7e8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:57.184748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653c213-5d81-4110-a086-c7d425f637fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:57.185604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653c213-5d81-4110-a086-c7d425f637fc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:57.185878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653c213-5d81-4110-a086-c7d425f637fc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:57.186155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653c213-5d81-4110-a086-c7d425f637fc None admin] PATH_INFO: `/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:57.187463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653c213-5d81-4110-a086-c7d425f637fc None admin] RBAC: Authorizing `identity:get_project(project_id=d423b6b2f1e9491d90a6fd2ea3357c53)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:57.188942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653c213-5d81-4110-a086-c7d425f637fc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:57.190661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 570/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:04:57 2022] GET /identity/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:57.200384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:57.312245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a6c4-8b15-48a4-b03d-a43a01d719b1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:57.314796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:57.346887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570ad4d-b9ab-4a93-aac6-dafc42337081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:57.347774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570ad4d-b9ab-4a93-aac6-dafc42337081 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:57.348059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570ad4d-b9ab-4a93-aac6-dafc42337081 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:57.348338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570ad4d-b9ab-4a93-aac6-dafc42337081 None admin] PATH_INFO: `/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53/users/b779fda5e5204e1987e6f9338e0eba0e/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:57.350431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570ad4d-b9ab-4a93-aac6-dafc42337081 None admin] RBAC: Authorizing `identity:create_grant(project_id=d423b6b2f1e9491d90a6fd2ea3357c53, user_id=b779fda5e5204e1987e6f9338e0eba0e, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:57.351917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570ad4d-b9ab-4a93-aac6-dafc42337081 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:57.368452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 583/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:04:57 2022] PUT /identity/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53/users/b779fda5e5204e1987e6f9338e0eba0e/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:59.354557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35eea-7246-4301-9d07-b81ad3a742d7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:59.355158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35eea-7246-4301-9d07-b81ad3a742d7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:59.355158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35eea-7246-4301-9d07-b81ad3a742d7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:59.356563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 571/1154] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:04:59.361991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:59.362445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:59.362550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:59.380381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:59.381585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:59.453987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 584/1155] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:04:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:04:59.485029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a35eea-7246-4301-9d07-b81ad3a742d7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:59.573135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:59.603235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a35eea-7246-4301-9d07-b81ad3a742d7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:59.641790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:59.642588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:59.642904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:59.643168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:59.654343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:04:59.662361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:04:59.663467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:04:59.664939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 572/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:04:59.673102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:59.775676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:59.784010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054c20c-8a4f-4407-a84a-de3f8773247d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:59.818891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:59.819600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:59.819885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:59.820193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:59.820949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:59.821256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:59.822764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:59.831722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 585/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:59.837961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:59.945387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591c88d-bf90-4e2d-bc3a-40c19b2d194b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:59.980708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:59.981563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:59.981833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:59.982100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] PATH_INFO: `/v3/users/b779fda5e5204e1987e6f9338e0eba0e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:59.983400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] RBAC: Authorizing `identity:get_user(user_id=b779fda5e5204e1987e6f9338e0eba0e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:59.984845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:59.988063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 573/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:04:59 2022] GET /identity/v3/users/b779fda5e5204e1987e6f9338e0eba0e => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:59.994036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:00.102579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1995d-b224-487a-822a-aadceb51b366 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:00.139153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:00.139804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:00.140088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:00.140366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:00.165476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:00.216815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:00.218213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:00.326499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 586/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:04:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:05:00.333519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:00.336816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:00.452236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d10ff3-aa9a-4220-a60d-2232dd24f2ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:00.487226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306dd61-c0db-4a08-8f61-632742299331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:00.488008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306dd61-c0db-4a08-8f61-632742299331 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:00.488282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306dd61-c0db-4a08-8f61-632742299331 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:00.488551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306dd61-c0db-4a08-8f61-632742299331 None admin] PATH_INFO: `/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:00.489763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306dd61-c0db-4a08-8f61-632742299331 None admin] RBAC: Authorizing `identity:get_project(project_id=d423b6b2f1e9491d90a6fd2ea3357c53)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:00.491251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306dd61-c0db-4a08-8f61-632742299331 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:00.492954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 574/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:05:00 2022] GET /identity/v3/projects/d423b6b2f1e9491d90a6fd2ea3357c53 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:00.500097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:00.585230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:00.637135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958dbe5-6eba-4cf2-a627-8ffe14ad22d0 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:00.672725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:00.673453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:00.673714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:00.673944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:00.676614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=d423b6b2f1e9491d90a6fd2ea3357c53, user.id=b779fda5e5204e1987e6f9338e0eba0e) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:00.676614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:00.676614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ea3b5-2ff0-4ae1-a05c-ef9f3d327bcb None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:00.682371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 587/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:05:00 2022] GET /identity/v3/role_assignments?user.id=b779fda5e5204e1987e6f9338e0eba0e&scope.project.id=d423b6b2f1e9491d90a6fd2ea3357c53&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:02.729988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911558c8-4919-4490-9a52-6b02ce1ef41f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:02.730488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911558c8-4919-4490-9a52-6b02ce1ef41f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:02.730488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911558c8-4919-4490-9a52-6b02ce1ef41f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:02.732047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 575/1162] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:02.737691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:02.738116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:02.738208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:02.757696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:02.758912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:02.832423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 588/1163] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:02.866869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911558c8-4919-4490-9a52-6b02ce1ef41f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:02.943425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:02.976288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911558c8-4919-4490-9a52-6b02ce1ef41f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:03.011112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:03.011749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:03.012029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:03.012317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:03.081401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:03.082887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:03.151616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:03.184648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 576/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:05:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:05:03.191656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:03.253086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:03.314961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9465cc-5f8f-49a5-8060-2987a5beb943 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:03.350213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:03.351042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:03.351322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:03.351595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:03.358740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:03.360299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-7a86d6f0-dbd3-447b-8f06-4b9db8055120 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:05:03.374200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:05:03.375410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:05:03.376434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 589/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:05:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:05:03.382467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:03.487357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12202820-b674-4a1c-b3ac-d15949c040f8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:03.522560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:03.523167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:03.523449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:03.523720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:03.524445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:03.524756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:03.526140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e47b10-4a48-43b0-af07-4df87a6ff677 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:03.533416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 577/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Jun 24 10:05:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:05.553261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc88722-0719-4721-a49f-cdb290b15a89 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:05.554148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc88722-0719-4721-a49f-cdb290b15a89 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:05.554148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc88722-0719-4721-a49f-cdb290b15a89 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:05.556415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 590/1167] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:05.561994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df396b-0c5e-456d-ad1f-472281fc70eb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:05.562421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df396b-0c5e-456d-ad1f-472281fc70eb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:05.562511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df396b-0c5e-456d-ad1f-472281fc70eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:05.581432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f5df396b-0c5e-456d-ad1f-472281fc70eb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:05.582628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f5df396b-0c5e-456d-ad1f-472281fc70eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:05.613668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:05.654039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 578/1168] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:05.678698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc88722-0719-4721-a49f-cdb290b15a89 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:05.807218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc88722-0719-4721-a49f-cdb290b15a89 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:05.817784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:05.849101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f481ca-323a-4d8b-98dc-16838c2e6bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:05.849766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f481ca-323a-4d8b-98dc-16838c2e6bef None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:05.850049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f481ca-323a-4d8b-98dc-16838c2e6bef None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:05.850330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f481ca-323a-4d8b-98dc-16838c2e6bef None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:05.851111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f481ca-323a-4d8b-98dc-16838c2e6bef None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:05.852492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f481ca-323a-4d8b-98dc-16838c2e6bef None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:05.869403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 591/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Fri Jun 24 10:05:05 2022] POST /identity/v3/domains => generated 267 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:05:07.911655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54044e0d-599e-416d-8112-30cece7f648b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:07.912233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54044e0d-599e-416d-8112-30cece7f648b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:07.912233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54044e0d-599e-416d-8112-30cece7f648b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:07.914228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 579/1170] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:07.919644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:07.920103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:07.920207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:07.939049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:07.940261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:08.014862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 592/1171] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:08.038997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54044e0d-599e-416d-8112-30cece7f648b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.143016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54044e0d-599e-416d-8112-30cece7f648b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.177790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236529f2-69be-4ff8-9727-1dcfde3e3d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.178640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236529f2-69be-4ff8-9727-1dcfde3e3d2c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.178946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236529f2-69be-4ff8-9727-1dcfde3e3d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.179204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236529f2-69be-4ff8-9727-1dcfde3e3d2c None admin] PATH_INFO: `/v3/domains/a53a68dd894d4aa28cb6cf76126d3510` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.190037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236529f2-69be-4ff8-9727-1dcfde3e3d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=a53a68dd894d4aa28cb6cf76126d3510)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.191527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236529f2-69be-4ff8-9727-1dcfde3e3d2c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.193148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 580/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Fri Jun 24 10:05:08 2022] GET /identity/v3/domains/a53a68dd894d4aa28cb6cf76126d3510 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:08.201913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.307454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:08.310469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdee71-9a60-4947-a40d-7690f6e714cb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.343889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5a722-66e0-40be-81f2-90d96bf5e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.344547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a722-66e0-40be-81f2-90d96bf5e98c None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.344869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a722-66e0-40be-81f2-90d96bf5e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.345175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a722-66e0-40be-81f2-90d96bf5e98c None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.345881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5a722-66e0-40be-81f2-90d96bf5e98c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.347370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5a722-66e0-40be-81f2-90d96bf5e98c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.377251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 593/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:05:08 2022] POST /identity/v3/projects => generated 361 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:05:10.439505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903cbc9-9ca7-4701-b6f1-5473488eeace None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:10.440462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903cbc9-9ca7-4701-b6f1-5473488eeace None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:10.440462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903cbc9-9ca7-4701-b6f1-5473488eeace None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:10.442655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 581/1174] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:10.451667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:10.452462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:10.452625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:10.477087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:10.478330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:10.571339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 594/1175] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:10.597246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903cbc9-9ca7-4701-b6f1-5473488eeace None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:10.635721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:10.725426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903cbc9-9ca7-4701-b6f1-5473488eeace None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:10.761136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3764fee-5796-4e64-9db7-23462b28ae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:10.761985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764fee-5796-4e64-9db7-23462b28ae53 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:10.762263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764fee-5796-4e64-9db7-23462b28ae53 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:10.762538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764fee-5796-4e64-9db7-23462b28ae53 None admin] PATH_INFO: `/v3/domains/a53a68dd894d4aa28cb6cf76126d3510` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:10.763879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3764fee-5796-4e64-9db7-23462b28ae53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a53a68dd894d4aa28cb6cf76126d3510)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:10.765333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3764fee-5796-4e64-9db7-23462b28ae53 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:10.767053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 582/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Fri Jun 24 10:05:10 2022] GET /identity/v3/domains/a53a68dd894d4aa28cb6cf76126d3510 => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:10.773297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:10.861871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:10.881197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faec02d-4b6d-4fc9-9573-1db3fe517933 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:10.916502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72285251-9bfc-4b82-8845-f5eb3947db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:10.917168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72285251-9bfc-4b82-8845-f5eb3947db98 None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:10.917449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72285251-9bfc-4b82-8845-f5eb3947db98 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:10.917719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72285251-9bfc-4b82-8845-f5eb3947db98 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:10.918434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72285251-9bfc-4b82-8845-f5eb3947db98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:10.919909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72285251-9bfc-4b82-8845-f5eb3947db98 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:10.943536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 595/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:05:10 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:05:13.028082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c379-5311-47f0-9a58-e1dd186a77f0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.028931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c379-5311-47f0-9a58-e1dd186a77f0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.028931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c379-5311-47f0-9a58-e1dd186a77f0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.031094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 583/1178] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:13.037430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.037881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.037972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.059415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:13.060613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:13.131449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 596/1179] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:13.155956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c379-5311-47f0-9a58-e1dd186a77f0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:13.262951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:13.273780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c379-5311-47f0-9a58-e1dd186a77f0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:13.309223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:13.309939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.310219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.310494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.321507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:05:13.328987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:05:13.330663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:05:13.331843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 584/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:05:13.337203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:13.419752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:13.455230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566bb9b-01c2-49a0-822a-882643e1c6af None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:13.490734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:13.491413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.491693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.491970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.492720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:13.493029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:13.494415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:13.502830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 597/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:13.509557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:13.623741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0e4cb-95f5-4c16-b4da-7e154fbd52d4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:13.659219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:13.660079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.660359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.660630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] PATH_INFO: `/v3/users/26e7d870051a4d7e954df3cb130cdbd7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.674475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] RBAC: Authorizing `identity:get_user(user_id=26e7d870051a4d7e954df3cb130cdbd7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:13.675966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:13.677617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 585/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:05:13 2022] GET /identity/v3/users/26e7d870051a4d7e954df3cb130cdbd7 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:13.687049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:13.798640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630e27f-08de-4c17-aea4-0f6473788634 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:13.827834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:13.835493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:13.836147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.836430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.836706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.911699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:13.913116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:14.027774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 598/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:05:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:05:14.037061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:14.051385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:14.144957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede69c2-79ca-4375-99e2-f0254e0b66f6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:14.180119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32192ea2-00ba-40a2-8d72-f6ba4b73381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:14.180891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32192ea2-00ba-40a2-8d72-f6ba4b73381c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:14.181167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32192ea2-00ba-40a2-8d72-f6ba4b73381c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:14.181437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32192ea2-00ba-40a2-8d72-f6ba4b73381c None admin] PATH_INFO: `/v3/projects/a9c03b8976554fe59bf98850c9e20976` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:14.182625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32192ea2-00ba-40a2-8d72-f6ba4b73381c None admin] RBAC: Authorizing `identity:get_project(project_id=a9c03b8976554fe59bf98850c9e20976)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:14.184103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32192ea2-00ba-40a2-8d72-f6ba4b73381c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:14.185781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 586/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:05:14 2022] GET /identity/v3/projects/a9c03b8976554fe59bf98850c9e20976 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:14.194241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:14.234022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:14.307673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5d3d8-9aa5-432d-810f-95014db129c3 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:14.348013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:14.348668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:14.348950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:14.349226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:14.350122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=a9c03b8976554fe59bf98850c9e20976, user.id=26e7d870051a4d7e954df3cb130cdbd7) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:14.350438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:14.351917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e7b09-fabb-4363-b648-71044d87ca34 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:14.358006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 599/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:05:14 2022] GET /identity/v3/role_assignments?user.id=26e7d870051a4d7e954df3cb130cdbd7&scope.project.id=a9c03b8976554fe59bf98850c9e20976&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:16.378160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37746faa-757f-46f0-a9a7-31a5b5113225 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:16.379114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37746faa-757f-46f0-a9a7-31a5b5113225 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:16.379114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37746faa-757f-46f0-a9a7-31a5b5113225 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:16.381303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 587/1186] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:16.386875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:16.387324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:16.387435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:16.406530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:16.408398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:16.483220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 600/1187] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:16.506764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37746faa-757f-46f0-a9a7-31a5b5113225 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:16.600731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:16.634736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37746faa-757f-46f0-a9a7-31a5b5113225 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:16.671449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:16.672229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:16.672508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:16.672794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:16.685194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:05:16.693137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:05:16.694197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:05:16.695226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 588/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:05:16.703070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:16.734234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:16.812825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a41e5-bbca-45a0-af20-84db098576c5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:16.851444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:16.852103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:16.852383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:16.852659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:16.853455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:16.853769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:16.855532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:16.864690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 601/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:05:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:16.870681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:16.980800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed0069-0c9a-4486-8571-caaea4699806 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:17.030188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:17.031072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:17.031375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:17.031650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] PATH_INFO: `/v3/users/26e7d870051a4d7e954df3cb130cdbd7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:17.033623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] RBAC: Authorizing `identity:get_user(user_id=26e7d870051a4d7e954df3cb130cdbd7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:17.035128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:17.036791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 589/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:05:16 2022] GET /identity/v3/users/26e7d870051a4d7e954df3cb130cdbd7 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:05:17.046516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:17.150512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:17.175634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b7295-2913-4240-b828-a4fb9b9f1eb2 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:17.215852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec278-5a32-47f2-8c5f-d10cccbf6063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:17.216719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec278-5a32-47f2-8c5f-d10cccbf6063 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:17.216999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec278-5a32-47f2-8c5f-d10cccbf6063 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:17.217267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec278-5a32-47f2-8c5f-d10cccbf6063 None admin] PATH_INFO: `/v3/projects/a9c03b8976554fe59bf98850c9e20976` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:17.218590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dec278-5a32-47f2-8c5f-d10cccbf6063 None admin] RBAC: Authorizing `identity:get_project(project_id=a9c03b8976554fe59bf98850c9e20976)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:17.220140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dec278-5a32-47f2-8c5f-d10cccbf6063 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:17.221842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 602/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:05:17 2022] GET /identity/v3/projects/a9c03b8976554fe59bf98850c9e20976 => generated 361 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:17.228290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:17.239592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:17.344490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae216e-63d2-46d8-a164-fcd5eab43db2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:17.380153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177839f-31b2-4d5a-b1ec-36e6cf461d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:17.381028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177839f-31b2-4d5a-b1ec-36e6cf461d0c None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:17.381302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177839f-31b2-4d5a-b1ec-36e6cf461d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:17.381577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177839f-31b2-4d5a-b1ec-36e6cf461d0c None admin] PATH_INFO: `/v3/projects/a9c03b8976554fe59bf98850c9e20976/users/26e7d870051a4d7e954df3cb130cdbd7/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:17.383729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177839f-31b2-4d5a-b1ec-36e6cf461d0c None admin] RBAC: Authorizing `identity:create_grant(project_id=a9c03b8976554fe59bf98850c9e20976, user_id=26e7d870051a4d7e954df3cb130cdbd7, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:17.385183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177839f-31b2-4d5a-b1ec-36e6cf461d0c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:17.401026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 590/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:05:17 2022] PUT /identity/v3/projects/a9c03b8976554fe59bf98850c9e20976/users/26e7d870051a4d7e954df3cb130cdbd7/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:05:19.391358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1356d2b-4b14-4e98-a26d-23693a3094dc None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:19.392022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1356d2b-4b14-4e98-a26d-23693a3094dc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:19.392022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1356d2b-4b14-4e98-a26d-23693a3094dc None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:19.393360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 603/1193] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:05:19.399338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02037b-6d93-4095-8b79-309423201385 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:19.400015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02037b-6d93-4095-8b79-309423201385 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:19.400204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02037b-6d93-4095-8b79-309423201385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:19.419831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1f02037b-6d93-4095-8b79-309423201385 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:19.421034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1f02037b-6d93-4095-8b79-309423201385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:19.486162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:19.495848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 591/1194] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:05:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:05:19.521904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1356d2b-4b14-4e98-a26d-23693a3094dc None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:19.643004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1356d2b-4b14-4e98-a26d-23693a3094dc None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:19.676629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:19.677411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:19.677686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:19.677955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:19.688399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:05:19.696394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:05:19.697483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:05:19.698346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 604/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:05:19.704066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02037b-6d93-4095-8b79-309423201385 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:19.810369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02037b-6d93-4095-8b79-309423201385 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:19.846844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:19.847496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:19.847773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:19.848041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:19.848784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:19.849079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:19.850456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:19.859071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 592/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:05:19.865663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:19.884038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:19.981074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc60d-2c23-48e8-8999-38f5a5ac7ea7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:20.017407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:20.018269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:20.018547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:20.018842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] PATH_INFO: `/v3/users/26e7d870051a4d7e954df3cb130cdbd7` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:20.020149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] RBAC: Authorizing `identity:get_user(user_id=26e7d870051a4d7e954df3cb130cdbd7)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:20.021589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:20.023364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 605/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Jun 24 10:05:19 2022] GET /identity/v3/users/26e7d870051a4d7e954df3cb130cdbd7 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:20.032673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:20.096461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:20.145594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2041fd-7607-4164-af6b-96e4fbbd507d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:20.181271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:20.181915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:20.182194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:20.182473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:20.259027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:20.260438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:20.319321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:20.368520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 593/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:05:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 10:05:20.374916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:20.463861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:20.486264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771aa90-b934-4ee0-8593-e16acf6f0933 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:20.522683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2a091-ad70-4056-9a49-0d3560be9449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:20.523501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a091-ad70-4056-9a49-0d3560be9449 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:20.523770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a091-ad70-4056-9a49-0d3560be9449 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:20.524034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a091-ad70-4056-9a49-0d3560be9449 None admin] PATH_INFO: `/v3/projects/a9c03b8976554fe59bf98850c9e20976` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:20.525310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a091-ad70-4056-9a49-0d3560be9449 None admin] RBAC: Authorizing `identity:get_project(project_id=a9c03b8976554fe59bf98850c9e20976)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:20.526750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a091-ad70-4056-9a49-0d3560be9449 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:20.528446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 606/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Jun 24 10:05:20 2022] GET /identity/v3/projects/a9c03b8976554fe59bf98850c9e20976 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:20.535276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:20.641405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd9f01-9e21-47a5-a699-82d4456a8b3c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:20.685006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:20.685636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:20.685912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:20.686187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:20.687046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=a9c03b8976554fe59bf98850c9e20976, user.id=26e7d870051a4d7e954df3cb130cdbd7) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:20.687353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:20.689292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bc055-292d-4155-a4cf-51e7a81494d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:20.695778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 594/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:05:20 2022] GET /identity/v3/role_assignments?user.id=26e7d870051a4d7e954df3cb130cdbd7&scope.project.id=a9c03b8976554fe59bf98850c9e20976&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:30.517503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=25 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:30.693092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:16.182013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed7624-5c69-4c93-8799-34020cbc2b4f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:16.182729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed7624-5c69-4c93-8799-34020cbc2b4f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:16.182729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed7624-5c69-4c93-8799-34020cbc2b4f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:16.184252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 607/1201] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:16.191988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:16.192552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:16.192552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:16.212144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:16.213550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:16.294882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 595/1202] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:16.324446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed7624-5c69-4c93-8799-34020cbc2b4f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:16.470723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed7624-5c69-4c93-8799-34020cbc2b4f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:16.512314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:16.513167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:16.513451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:16.513721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:16.525603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:16.527234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:16.545680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:16.546863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:16.547867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:16.547867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:16.547867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:16.547867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:16.547867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 608/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:16.555189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:16.690086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1669aec-1bcd-4a08-8caa-b539e41ddab8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:16.733994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:16.748971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:16.749744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:16.750028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:16.750359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:16.751131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:16.751503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:16.753039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecafbbb-f5de-4ebd-9a84-d27a1b0f15ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:16.771578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 596/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:16.777565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:16.851880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:16.917277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe89acb-44aa-4a36-8570-db7d175ba87e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:16.968356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dc947-9284-4755-9778-c7930658feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:16.969057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc947-9284-4755-9778-c7930658feda None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:16.969342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc947-9284-4755-9778-c7930658feda None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:16.969723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc947-9284-4755-9778-c7930658feda None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:16.970549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dc947-9284-4755-9778-c7930658feda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:16.972160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dc947-9284-4755-9778-c7930658feda None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:16.999881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 609/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Jun 24 10:06:16 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:06:19.095785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50249-3e7f-43e8-8b49-0553d6717495 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:19.096343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50249-3e7f-43e8-8b49-0553d6717495 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:19.096343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf50249-3e7f-43e8-8b49-0553d6717495 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:19.097638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 597/1206] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:19.103480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975160a-90c0-4342-a370-d39531770a28 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:19.104323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975160a-90c0-4342-a370-d39531770a28 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:19.104534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975160a-90c0-4342-a370-d39531770a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:19.123909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4975160a-90c0-4342-a370-d39531770a28 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:19.125163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4975160a-90c0-4342-a370-d39531770a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:19.206560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 610/1207] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:19.232058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf50249-3e7f-43e8-8b49-0553d6717495 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:19.348342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf50249-3e7f-43e8-8b49-0553d6717495 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:19.388035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:19.388990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:19.389282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:19.389567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:19.403673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:06:19.411513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:19.412827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:06:19.413784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:19.419018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 598/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:19.419146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975160a-90c0-4342-a370-d39531770a28 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:19.468130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:19.542688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975160a-90c0-4342-a370-d39531770a28 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:19.581372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:19.582101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:19.582391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:19.582698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:19.583646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:19.584004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:19.585640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:19.596323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 611/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:19.602724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:19.660482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:19.728683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb93d84-9510-4a8f-8441-2969f96f10b8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:19.770025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:19.773939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:19.774232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:19.774540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:19.787217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:19.788821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:19.807065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:19.808371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:19.809347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:19.809347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:19.809347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:19.809347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:19.809347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 599/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:19.814651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:19.928477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:19.931551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a83601-4e69-43a4-8992-05100e17cae8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:19.971377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:19.972143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:19.972456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:19.972774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:19.973535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:19.973840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:19.975299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:19.996207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 612/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:19.997954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:20.115098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa89bc-ba0d-4978-8240-505834337b7f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:20.117118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:20.155712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:20.156665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:20.156910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:20.157227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:20.165672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:20.167349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:06:20.183391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:20.184709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:20.185682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 600/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:20.191191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:20.310330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f651a4-a095-4bf8-a217-60d63b8bd228 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:20.349037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:20.349726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:20.350010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:20.350296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:20.351195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:20.351532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:20.353011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:20.371651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 613/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Jun 24 10:06:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:20.377722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:20.497786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca52a-c14d-436d-a98b-329c08c1fdb6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:20.539295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:20.540161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:20.540604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:20.540965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:20.581163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:20.630893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:20.632680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:20.757255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 601/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:06:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:06:20.763373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:20.786318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:20.883161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3936546-20f8-4262-8b12-2c0363dbef4f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:20.922720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:20.923803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:20.924092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:20.924370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:20.936678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:20.938250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:20.959287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:20.960399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:20.966071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:20.966071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:20.966071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:20.966071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:20.966071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 614/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:20.969109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:21.042429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:21.088895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6779c-cc8d-48b7-8f35-0e79da3b0ca7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:21.131831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:21.132589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:21.132814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:21.133090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:21.133851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:21.134146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:21.135671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:21.148496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 602/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:21.154360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:21.239041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:21.276464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fe1c4-a8d3-4a23-95a7-4427bef10d4d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:21.315006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:21.315731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:21.316017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:21.316297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:21.324150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:21.325649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:06:21.343261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:21.344646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:21.345757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:21.349502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 615/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:06:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:06:21.351022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:21.472294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7ce2f-acfa-43d8-9098-a0da2ac58e92 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:21.498031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:21.510389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:21.511384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:21.511721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:21.512064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:21.512935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:21.513317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:21.514908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4bc8c-8d3c-4c18-9ce7-7638791522a1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:21.533705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 603/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:06:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:06:21.535073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:21.658617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299ac60-a9be-4043-bb82-9a89ce4fa918 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:21.695952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:21.702983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:21.703803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:21.710325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:21.711120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:21.712123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=edd302416f834b5b9b2f1a8c9955349d) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:21.712937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:21.714971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb1c94-4723-400c-bad8-5ae242c36ef8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:21.730733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 616/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:06:21 2022] GET /identity/v3/role_assignments?user.id=edd302416f834b5b9b2f1a8c9955349d&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:23.848911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f94cc-1cb9-4613-9f54-5c1caeb68b63 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:23.849480 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f94cc-1cb9-4613-9f54-5c1caeb68b63 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:23.849480 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f94cc-1cb9-4613-9f54-5c1caeb68b63 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:23.850890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 604/1220] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:23.856984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:23.857384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:23.857537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:23.875056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:23.876617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:23.950210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 617/1221] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:23.975542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f94cc-1cb9-4613-9f54-5c1caeb68b63 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:24.099925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f94cc-1cb9-4613-9f54-5c1caeb68b63 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:24.138857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:24.139699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:24.140021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:24.140289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:24.152616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:06:24.160973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:24.162142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:24.163212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 605/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:06:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:24.168857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:24.283007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f0df8-5322-4d40-89df-760f7829e95c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:24.310825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:24.328760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:24.329526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:24.329802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:24.330077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:24.330956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:24.331241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:24.332678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:24.341574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 618/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:24.347590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:24.397744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:24.465075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd784c-d8b0-400b-adc9-241b85a3454e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:24.501408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:24.502231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:24.502526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:24.502818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:24.513881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:24.515381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:24.533678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:24.534835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:24.535944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:24.535944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:24.535944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:24.535944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:24.535944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 606/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:24.540725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:24.652444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73945707-7842-409b-a9fc-14dbb3f420c7 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:24.691066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:24.691742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:24.692023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:24.692310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:24.693022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:24.693331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:24.694733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:24.707363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 619/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:24.712918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:24.812362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:24.820702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9553-46d0-4e5a-9c81-eefb4640c09a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:24.859619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:24.860434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:24.860901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:24.861507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:24.869224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:24.870669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:06:24.885699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:24.887033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:24.888233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:24.890894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 607/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:06:24.893039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:24.946559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:25.003022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512e71f-99c6-48ef-9641-b9f78aca1a56 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:25.047470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:25.048421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:25.048796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:25.049331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:25.050517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:25.050888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:25.052343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:25.075355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 620/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Jun 24 10:06:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:25.076538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:25.183118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09bcb7-dba3-4e45-ad93-b29723df0ebb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:25.219199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:25.220079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:25.220473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:25.220786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:25.233785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:25.235620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:25.255536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:25.256484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:25.257464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:25.257464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:25.257464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:25.257464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:25.257464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 608/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:25.263383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:25.375464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c3db8-aae1-4b19-9c42-facc81a1f1c9 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:25.377960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:25.416757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:25.417428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:25.417708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:25.417976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:25.418698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:25.419095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:25.420528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:25.433412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 621/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:25.440807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:25.451548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:25.557106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e6a8f-10f7-4035-91a3-cc30682c9a49 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:25.594100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:25.595075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:25.595391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:25.595831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:25.603413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:25.605426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:06:25.621346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:25.622419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:25.623420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 609/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:25.630255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:25.752888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc774f-74b0-4c58-bf87-c66651265428 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:25.790357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:25.791168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:25.791448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:25.791770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:25.792659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:25.792959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:25.794514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea4e7e-4457-4a4e-8831-6f8ecc32195c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:25.808460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 622/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:25.813998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:25.886638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:25.927228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21958d29-a26c-42e1-bcff-885142b281e4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:25.965540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d1292-4a67-42f3-b999-f7f8ab1bdc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:25.966493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d1292-4a67-42f3-b999-f7f8ab1bdc56 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:25.966825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d1292-4a67-42f3-b999-f7f8ab1bdc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:25.967144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d1292-4a67-42f3-b999-f7f8ab1bdc56 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/edd302416f834b5b9b2f1a8c9955349d/roles/1a2e60175b5a4288bd126b2693287c7d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:25.983608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d1292-4a67-42f3-b999-f7f8ab1bdc56 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=edd302416f834b5b9b2f1a8c9955349d, role_id=1a2e60175b5a4288bd126b2693287c7d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:25.985213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d1292-4a67-42f3-b999-f7f8ab1bdc56 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:25.998590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 610/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:06:25 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/edd302416f834b5b9b2f1a8c9955349d/roles/1a2e60175b5a4288bd126b2693287c7d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:06:28.102116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c75c-863d-40b6-8731-6e598df44731 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:28.102644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c75c-863d-40b6-8731-6e598df44731 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:28.102644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c75c-863d-40b6-8731-6e598df44731 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:28.104506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 623/1233] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:28.110751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:28.110751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:28.110751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:28.130385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:28.131984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:28.207526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 611/1234] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:28.235766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c75c-863d-40b6-8731-6e598df44731 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:28.261779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:28.343919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c75c-863d-40b6-8731-6e598df44731 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:28.379241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:28.380059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:28.380342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:28.380618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:28.392009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:06:28.401099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:28.402217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 10:06:28.403288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:28.406881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 624/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Jun 24 10:06:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:28.410233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:28.471026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:28.537806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e140022-4aa0-40d1-bd39-70a8cbca9eba None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:28.576375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:28.577049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:28.577355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:28.577631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:28.578379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:28.578683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:28.580158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:28.590097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 612/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Jun 24 10:06:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:28.596070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:28.686618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:28.705567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509eca-e930-4118-a6fc-0faf0f927a73 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:28.743226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:28.744110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:28.744452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:28.744783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:28.760593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:28.762154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:28.780674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:28.781774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:28.783294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:28.783294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:28.783294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:28.783294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:28.783294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 625/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:06:28.787940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:28.903416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa454f-5c59-46df-9c7f-57b9dd666b33 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:28.913351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:28.941267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:28.942010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:28.942382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:28.942726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:28.943484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:28.943793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:28.945229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:28.959442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 613/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:28.965731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:29.087275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2af1a8-c82f-4744-a2ef-69e28a28136e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:29.124637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:29.125459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:29.125738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:29.126011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:29.129430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:29.135102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:29.136933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:06:29.151555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:29.153600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:29.155504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 626/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:06:29.160944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:29.267973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b31c0c-c35d-4776-815a-361cf41ba944 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:29.305372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:29.306058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:29.306343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:29.306619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:29.307498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:29.307811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:29.309425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:29.330465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 614/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Jun 24 10:06:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:06:29.336603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:29.473605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a3cd-5984-4d4b-9697-cdce10fe337e None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:29.512361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:29.513150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:29.513441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:29.513721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:29.577386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:29.584671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:29.586172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:29.698755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 627/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:06:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:06:29.699673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:29.735901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:29.821054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbec7f-3f42-4621-9b48-08fe4f6fe9b0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:29.859911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:29.860752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:29.861135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:29.861700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:29.872671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:29.874133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:29.892558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:29.893698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:29.894850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:29.894850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:29.894850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:29.894850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:29.894850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 615/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:29.899640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:30.011803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:30.019473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59640ce-ecf7-47f8-beae-b913ddb9be6b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:30.057151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:30.057835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:30.058119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:30.058397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:30.059158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:30.059466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:30.060851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:30.073244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 628/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:06:30.079081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:30.165255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:30.188406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bed15a-5e40-46e9-bca2-4f86617e1286 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:30.224588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:30.225491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:30.225799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:30.226069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:30.234629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:30.236680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:06:30.252998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:30.254172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:30.255151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 616/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:06:30.261461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:30.370321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49eb6cf-e30a-4ba4-ba82-00669c0e5b05 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:30.415012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:30.415748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:30.415972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:30.416245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:30.417044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:30.417413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:30.418877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ee2e-f5d1-4b64-800d-d0b43fecb9db None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:30.433744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 629/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:30.440771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:30.554454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad3685-df70-4451-9a3c-461f1de86dcc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:30.591094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:30.591792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:30.592074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:30.592382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:30.593251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] RBAC: Adding query filter params (role.id=1a2e60175b5a4288bd126b2693287c7d, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=edd302416f834b5b9b2f1a8c9955349d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:30.593656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:30.595732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29a295-5406-4b7a-b0a3-be1c25d1e23b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:30.603083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 617/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:06:30 2022] GET /identity/v3/role_assignments?user.id=edd302416f834b5b9b2f1a8c9955349d&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=1a2e60175b5a4288bd126b2693287c7d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:06:32.745777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1253f13-3bbf-40e1-b535-8953a5cc5fe1 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:32.746339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1253f13-3bbf-40e1-b535-8953a5cc5fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:32.746339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1253f13-3bbf-40e1-b535-8953a5cc5fe1 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:32.747842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 630/1247] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:32.753936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:32.754421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:32.754608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:32.760461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:32.773641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:32.774960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:32.844358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 618/1248] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:32.868505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1253f13-3bbf-40e1-b535-8953a5cc5fe1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:32.887236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:32.987973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1253f13-3bbf-40e1-b535-8953a5cc5fe1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:33.028380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:33.030544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:33.030544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:33.030544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:33.041137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:06:33.048919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:33.050405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:33.051834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 631/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:33.056390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:33.162815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb24ce-4aac-461e-b197-c0d8b4040404 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:33.196708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:33.203675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:33.203675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:33.203910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:33.204185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:33.205325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:33.205633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:33.207095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:33.216034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 619/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:33.222177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:33.311733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:33.342415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbc7f3-a99d-4aae-a330-666bafb3f8e1 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:33.381230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:33.382166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:33.382447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:33.382747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:33.395388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:33.397147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:33.417894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:33.419175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:33.424996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:33.424996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:33.424996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:33.424996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:33.424996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 632/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:33.425498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:33.545631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a06be-3df8-416b-9d1e-35ba01e6890c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:33.586013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:33.586652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:33.586957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:33.587273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:33.588028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:33.588332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:33.589738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:33.603336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 620/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:33.609696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:33.723575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedbcbd-9d03-432e-968a-432792cfd018 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:33.753097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:33.761799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:33.762610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:33.762915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:33.763186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:33.772500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:33.775909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:06:33.795966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:33.797118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:33.798062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 633/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:33.804254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:33.874071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:33.926222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf54d8a-f372-4ac0-9d95-9d93d31ab62c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:33.969840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:33.970889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:33.971173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:33.971447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:33.972406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:33.972769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:33.974331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:33.993666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 621/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Jun 24 10:06:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:34.000127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:34.119478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f709e-8e6c-4b61-bf78-f7046e1f23d8 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:34.156159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:34.156728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:34.157010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:34.157304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:34.219034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:34.234927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:34.236829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:34.358212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 634/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:06:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 10:06:34.360177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:34.486430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d0e8-7359-4480-b1d5-6776a005fa60 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:34.500780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:34.526932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:34.527951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:34.528271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:34.528670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:34.541397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:34.543232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:34.562396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:34.564144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:34.565150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:34.565150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:34.565150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:34.565150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:34.565150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 622/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:34.571309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:34.670864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:34.684640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387483b-83a6-46b4-8deb-b6084197ee60 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:34.725085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:34.725786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:34.726098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:34.726453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:34.727309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:34.727618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:34.729471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:34.743574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 635/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:06:34.749628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:34.860901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d0335-4d59-415f-b807-15adf1e6bf65 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:34.900801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:34.901608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:34.901887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:34.902159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:34.909378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:34.911189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:06:34.927168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:34.928139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:34.929453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 623/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:34.934794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:35.053088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ae0d1-c19c-42a7-bce2-8dac712ba0c6 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:35.090258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:35.091221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:35.091533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:35.091864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:35.092860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:35.093239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:35.094926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74127086-2e98-4374-b8ff-f1d56186483e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:35.110820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 636/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:06:35.117314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:35.147344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:35.238160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca68f0-b6a1-46d3-84f8-25a12ee28dda None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:35.275073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:35.275731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:35.276017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:35.276278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:35.277111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=edd302416f834b5b9b2f1a8c9955349d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:35.277418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:35.278887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fed61-2fc3-4c40-afa3-9f843a6d4006 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:35.284897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 624/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:06:35 2022] GET /identity/v3/role_assignments?user.id=edd302416f834b5b9b2f1a8c9955349d&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=b10ab4271e1845e4aefab70d82a12def => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:37.405489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e5167-2264-4053-9fd0-01b9bb1d01eb None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:37.406066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e5167-2264-4053-9fd0-01b9bb1d01eb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:37.406066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e5167-2264-4053-9fd0-01b9bb1d01eb None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:37.407500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 637/1261] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:37.413481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:37.414032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:37.414258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:37.434694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:37.436075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:37.508123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 625/1262] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:37.540680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e5167-2264-4053-9fd0-01b9bb1d01eb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:37.553449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:37.654174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e5167-2264-4053-9fd0-01b9bb1d01eb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:37.693162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:37.693877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:37.694158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:37.694431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:37.705514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:06:37.713081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:37.714129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:37.715028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 638/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:37.721627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:37.741928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:37.829985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5b97b-a9e6-4845-8dfa-31e940646f16 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:37.866426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:37.867116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:37.867397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:37.867668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:37.868423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:37.868719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:37.870108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:37.878357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 626/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:37.884370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:37.975112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:38.017242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d00c-d435-4dd6-a88f-9db48cdbfce5 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:38.056612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:38.057407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:38.057692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:38.057967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:38.068889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:38.070347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:38.087501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:38.089188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:38.090872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:38.090872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:38.090872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:38.090872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:38.090872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 639/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:38.096716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:38.188312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:38.221401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f010c1-89f0-4681-b342-4f5da97f1155 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:38.260335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:38.260909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:38.261186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:38.261459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:38.262187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:38.262490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:38.263912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:38.275996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 627/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:38.283343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:38.414004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d64a6-0798-46cd-bb9c-7ebdbda7c34b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:38.437049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:38.455038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:38.456026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:38.456337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:38.456615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:38.464396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:38.469839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:06:38.493379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:38.494562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:38.495680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 640/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:38.503175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:38.621057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14843c80-e64f-4d4d-bad7-ea84e85981d5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:38.656537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:38.657196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:38.657479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:38.657775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:38.658612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:38.658946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:38.660397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:38.669035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:38.678516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 628/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Jun 24 10:06:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:38.684650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:38.796971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba1fc8-352f-4a00-af73-bf910f68c368 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:38.832828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:38.833613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:38.833887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:38.834158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:38.845911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:38.847388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:38.866056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:38.867091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:38.868018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:38.868018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:38.868018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:38.868018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:38.868018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 641/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 10:06:38.873214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:38.980046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79951382-b2b7-4a98-bdfc-1647740185ec None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:39.015188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:39.015833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:39.016112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:39.016386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:39.017133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:39.017433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:39.018844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:39.030043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 629/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:39.036578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:39.079221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:39.150265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80879257-838a-45fd-b9c4-07cb53916eda None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:39.185357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:39.186129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:39.186409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:39.186682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:39.194103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:39.195582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:06:39.210131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:39.211235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:39.212127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 642/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:39.219959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:39.288375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:39.339088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9f1e0-88c2-4ea1-a4e7-ac9d386964a4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:39.377194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:39.377836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:39.378112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:39.378382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:39.379192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:39.379495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:39.380928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282d85-b40e-4603-a95e-73705462c110 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:39.394427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 630/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:39.403525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:39.504386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:39.515222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217572f-0012-45c6-9c57-bad1d23689dd None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:39.550849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8725f-3086-4d2e-9760-055ed582ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:39.551724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8725f-3086-4d2e-9760-055ed582ae40 None admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:39.552005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8725f-3086-4d2e-9760-055ed582ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:39.552280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8725f-3086-4d2e-9760-055ed582ae40 None admin] PATH_INFO: `/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/edd302416f834b5b9b2f1a8c9955349d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:39.554526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8725f-3086-4d2e-9760-055ed582ae40 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e4a8e6312d4526bed0a72bce3741b7, user_id=edd302416f834b5b9b2f1a8c9955349d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:39.556013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8725f-3086-4d2e-9760-055ed582ae40 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:39.572688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 643/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Jun 24 10:06:39 2022] PUT /identity/v3/projects/f7e4a8e6312d4526bed0a72bce3741b7/users/edd302416f834b5b9b2f1a8c9955349d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:06:41.652046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd800c1-191b-4b0e-a5ab-7ea14e91c4d9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:41.652719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd800c1-191b-4b0e-a5ab-7ea14e91c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:41.652719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd800c1-191b-4b0e-a5ab-7ea14e91c4d9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:41.654014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 631/1274] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:41.660467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:41.661244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:41.661379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:41.684351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:41.685554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:41.758274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 644/1275] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:41.782241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd800c1-191b-4b0e-a5ab-7ea14e91c4d9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:41.892381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd800c1-191b-4b0e-a5ab-7ea14e91c4d9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:41.910631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:41.931519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:41.932303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:41.932588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:41.932865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:41.945127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 10:06:41.952898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:41.954106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:41.955459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 632/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Jun 24 10:06:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:41.960887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:42.049149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:42.068477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9bd3f-7572-4f55-86b2-95205f302b02 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:42.103534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:42.104551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:42.104832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:42.105108 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:42.105857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:42.106158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:42.107583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:42.116121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 645/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Jun 24 10:06:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:42.122190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:42.230819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41044b6-2935-46c9-ac12-91bcf0fae1d0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:42.271492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:42.272274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:42.272550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:42.272824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:42.284331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:42.285806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:42.303038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:42.304041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:42.304929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:42.304929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:42.304929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:42.304929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:42.304929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 633/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:42.309898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:42.417324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a530c1b-01f3-4b05-8983-3e0909a7184d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:42.456595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:42.457263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:42.457545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:42.457822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:42.458545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:42.458898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:42.460288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:42.472711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 646/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:42.481690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:42.515452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:42.589436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421b322-01ab-4c53-93ae-ee83b568a45e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:42.625176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:42.625965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:42.626389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:42.627152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:42.635559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:42.637032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 10:06:42.651800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:42.653400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:42.655017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 634/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:42.660624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:42.664717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:42.769233 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacac11-0fbc-4697-9a7a-36f759da441d None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:42.805295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:42.805968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:42.806252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:42.806526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:42.807414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:42.807731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:42.809175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:42.826719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 647/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Jun 24 10:06:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:42.833470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:42.936759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:42.948095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2810ff-c63a-4d28-992d-72b26ea643df None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:42.983975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:42.984613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:42.984890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:42.985162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:43.060520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:43.062853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:43.153511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:43.171488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 635/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Jun 24 10:06:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 10:06:43.178282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:43.244623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:43.286511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348db27-ed35-45e3-b1c6-e0578346152b None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:43.323112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:43.323835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:43.324116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:43.324387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:43.334813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:43.336270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 10:06:43.353804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:43.355171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:43.356189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 10:06:43.356189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 10:06:43.356189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 10:06:43.356189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:43.356189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 648/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:43.361991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:43.478697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c253e11-e054-41ab-8ee9-b5e3cd4394d8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:43.520937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:43.521888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:43.522258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:43.522635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:43.523795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:43.524219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:43.526163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:43.540363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 636/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Jun 24 10:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:43.546472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:43.673814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118688d-2eac-463e-9f17-40f3114d32db None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:43.688071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:43.711224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:43.712036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:43.712320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:43.712597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:43.720456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:43.721910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 10:06:43.736146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:43.737161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: ERROR keystone.server.flask.application Jun 24 10:06:43.738213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 649/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Jun 24 10:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:43.745744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:43.812685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:43.867070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9c1db-d169-45a0-a728-fb885ea49de1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:43.902102 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:43.902756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:43.903060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:43.903338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:43.904125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:43.904436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:43.905891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cee862-4726-45a0-a9bc-f169971b4e2a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:43.918150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 637/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Jun 24 10:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:43.926843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:44.050884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79de092-f5d1-4bc5-aa6a-3f5337ff28b6 None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:44.089488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:44.090174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:44.090463 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:44.090753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:44.091593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] RBAC: Adding query filter params (role.id=b10ab4271e1845e4aefab70d82a12def, scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7, user.id=edd302416f834b5b9b2f1a8c9955349d) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:44.091899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:44.093378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be3666-6768-4f8b-b53a-30473be61a5e None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:44.099690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 650/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Jun 24 10:06:43 2022] GET /identity/v3/role_assignments?user.id=edd302416f834b5b9b2f1a8c9955349d&scope.project.id=f7e4a8e6312d4526bed0a72bce3741b7&role.id=b10ab4271e1845e4aefab70d82a12def => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:46.162978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1397-aeb6-4e72-8b23-a959dacd28ca None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:46.163897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1397-aeb6-4e72-8b23-a959dacd28ca None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:46.163897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1397-aeb6-4e72-8b23-a959dacd28ca None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:46.166125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 638/1288] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:46.172378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:46.173232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:46.173413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:46.192110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:46.193323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:46.226012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:46.265939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 651/1289] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:46.289297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1397-aeb6-4e72-8b23-a959dacd28ca None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:46.398186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1397-aeb6-4e72-8b23-a959dacd28ca None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:46.409560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:46.438279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:46.439072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:46.439351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:46.439618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:46.440279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:46.441836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:46.454165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:46.455413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:46.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 639/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:06:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:46.462831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:46.574147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50e86c-475a-4d18-bff0-17d23500d9c5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:46.609853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:46.610515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:46.610853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:46.611130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:46.611842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:46.612141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:46.613524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120ee2a-993a-47d6-8b85-1a63bdeac9d8 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:46.618460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 652/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Jun 24 10:06:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:46.624597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:46.748291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0f59-a31f-4b91-af9c-43bad1af2494 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:46.783077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:46.783736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:46.784018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:46.784296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:46.785007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:46.785315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:46.786731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6842c3b-7358-46b2-8caa-23b0d7381d80 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:46.791382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 640/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Jun 24 10:06:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:48.795954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30bcbc-66c2-4576-a38e-d213faf7f853 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:48.796784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30bcbc-66c2-4576-a38e-d213faf7f853 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:48.796946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30bcbc-66c2-4576-a38e-d213faf7f853 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:48.799225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 653/1293] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:48.804578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bca14-cd71-46c1-917b-58a9d102845d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:48.805028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bca14-cd71-46c1-917b-58a9d102845d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:48.805122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bca14-cd71-46c1-917b-58a9d102845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:48.823418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-132bca14-cd71-46c1-917b-58a9d102845d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:48.824622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-132bca14-cd71-46c1-917b-58a9d102845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:48.855129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:48.909716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 641/1294] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:48.934520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30bcbc-66c2-4576-a38e-d213faf7f853 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:48.969777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:49.040826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30bcbc-66c2-4576-a38e-d213faf7f853 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:49.076918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a38a45-adaf-4c43-9c93-44c82e32160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:49.077578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38a45-adaf-4c43-9c93-44c82e32160c None admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:49.077860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38a45-adaf-4c43-9c93-44c82e32160c None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:49.078133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38a45-adaf-4c43-9c93-44c82e32160c None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:49.078882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a38a45-adaf-4c43-9c93-44c82e32160c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:49.080335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a38a45-adaf-4c43-9c93-44c82e32160c None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:49.094923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 654/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Jun 24 10:06:48 2022] POST /identity/v3/services => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:06:51.129694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a02f-c4b0-4543-b227-160dcd0b0a5b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:51.130262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a02f-c4b0-4543-b227-160dcd0b0a5b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:51.130262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a02f-c4b0-4543-b227-160dcd0b0a5b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:51.132438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 642/1296] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:51.141073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:51.141473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:51.141719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:51.165612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:51.166834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:51.245042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 655/1297] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 10:06:51.269429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a02f-c4b0-4543-b227-160dcd0b0a5b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:51.377952 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a02f-c4b0-4543-b227-160dcd0b0a5b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:51.416565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:51.417575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:51.417918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:51.418317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:51.419351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:51.421603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:51.434786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:51.436010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:51.436882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 643/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:06:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:51.444496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:51.470682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:51.569088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52d868-ef10-4a2e-89a7-8cb1d3b3f8e0 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:51.608233 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:51.608887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:51.609168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:51.609439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:51.610150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:51.610486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:51.612447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:51.617714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 656/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Jun 24 10:06:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:51.623894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:51.658358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:51.734754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0acfc1-2e16-4097-98b3-1595be410504 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:51.769805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:51.770454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:51.770748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:51.771057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:51.771849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] RBAC: Adding query filter params (interface=public, service_id=ac9a22c344e74185abc04c273aeb5263, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:51.772158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:51.773557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205802b2-70cc-4410-abba-5597366fbd73 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:51.778737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 644/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Jun 24 10:06:51 2022] GET /identity/v3/endpoints?service_id=ac9a22c344e74185abc04c273aeb5263&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:51.786228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:51.866492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:51.889473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237af9b-f043-48ed-a40c-4f84249fd60f None admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:51.926532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ced4d-192f-40b1-b266-39be4c73d7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:51.927243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ced4d-192f-40b1-b266-39be4c73d7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:51.927521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ced4d-192f-40b1-b266-39be4c73d7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:51.927797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ced4d-192f-40b1-b266-39be4c73d7e1 None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:51.928494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ced4d-192f-40b1-b266-39be4c73d7e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:51.929889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ced4d-192f-40b1-b266-39be4c73d7e1 None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:51.937613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 657/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Jun 24 10:06:51 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:06:53.981884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46c083-8109-4c95-afd3-984e42e051e8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:53.982779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46c083-8109-4c95-afd3-984e42e051e8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:53.982779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46c083-8109-4c95-afd3-984e42e051e8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:53.984909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 645/1302] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:06:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:06:53.992882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:53.993323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:53.993412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:54.013502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:06:54.014681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:06:54.085056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 658/1303] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:06:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 10:06:54.109043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46c083-8109-4c95-afd3-984e42e051e8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:54.218986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:54.227575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46c083-8109-4c95-afd3-984e42e051e8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:54.264919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:54.265642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:54.265929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:54.266203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:54.267153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:54.268350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 10:06:54.288642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 10:06:54.290363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Jun 24 10:06:54.292195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 646/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Jun 24 10:06:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 10:06:54.300271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:54.386470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:54.422511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca170aa-f9b9-4d65-bdcc-da733fcca303 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:54.458857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:54.459512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:54.459800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:54.460074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] PATH_INFO: `/v3/services` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:54.460785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:06:54.461099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:54.462475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41fbc-5dc6-4f19-91b0-2eeea3f1ba0d None admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:54.467791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 659/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Jun 24 10:06:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:06:54.476350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:06:54.584289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13191a-394f-4d90-abf6-153969ef7405 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:06:54.619340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c5cc6-5415-4d5f-89b2-fdc3b1c4fd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:06:54.619969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c5cc6-5415-4d5f-89b2-fdc3b1c4fd2f None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:06:54.620245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c5cc6-5415-4d5f-89b2-fdc3b1c4fd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:06:54.620498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c5cc6-5415-4d5f-89b2-fdc3b1c4fd2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:06:54.621228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c5cc6-5415-4d5f-89b2-fdc3b1c4fd2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:06:54.622594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c5cc6-5415-4d5f-89b2-fdc3b1c4fd2f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:06:54.647307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 647/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Jun 24 10:06:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:07:04.464722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=30 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:07:04.641584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:07:57.870603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b38aa-0754-4714-ada0-cc86d5478ea4 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:07:57.871198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b38aa-0754-4714-ada0-cc86d5478ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:07:57.871198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b38aa-0754-4714-ada0-cc86d5478ea4 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:07:57.876723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 660/1307] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:07:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:07:57.878232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdabcf-dfed-40cc-85e7-84d9c1312cfc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:07:57.878790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdabcf-dfed-40cc-85e7-84d9c1312cfc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:07:57.879123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdabcf-dfed-40cc-85e7-84d9c1312cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:07:58.022316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-acfdabcf-dfed-40cc-85e7-84d9c1312cfc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:07:58.029710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-acfdabcf-dfed-40cc-85e7-84d9c1312cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:07:58.214406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 648/1308] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:07:58.270480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ff0f-dbf7-479c-8282-bcaeba68204b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:07:58.270792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ff0f-dbf7-479c-8282-bcaeba68204b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:07:58.271474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ff0f-dbf7-479c-8282-bcaeba68204b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:07:58.273291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 661/1309] 10.222.0.64 () {54 vars in 964 bytes} [Fri Jun 24 10:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:07:58.279768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eeb5c-b26b-48b9-873c-219a535ea719 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:07:58.280052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eeb5c-b26b-48b9-873c-219a535ea719 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:07:58.280347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eeb5c-b26b-48b9-873c-219a535ea719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:07:58.364252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-313eeb5c-b26b-48b9-873c-219a535ea719 None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:07:58.365487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-313eeb5c-b26b-48b9-873c-219a535ea719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:07:58.388572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:07:58.451970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 649/1310] 10.222.0.64 () {58 vars in 1061 bytes} [Fri Jun 24 10:07:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:07:58.461310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ff0f-dbf7-479c-8282-bcaeba68204b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:07:58.463124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3db2ff0f-dbf7-479c-8282-bcaeba68204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:07:58.584607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ff0f-dbf7-479c-8282-bcaeba68204b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:07:58.662010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:07:58.662571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:07:58.662888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:07:58.663163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:07:58.664793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:07:58.728079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:07:58.790385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:07:58.791873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307889f-b014-464e-a269-4049a33394e0 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:07:58.814499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 662/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:01.118630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f91286-f148-4e9d-b539-f6059b43f13f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:01.119269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f91286-f148-4e9d-b539-f6059b43f13f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:01.119269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f91286-f148-4e9d-b539-f6059b43f13f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:01.120487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 650/1312] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:01.125844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987398af-55ef-4cc6-a366-180d3120b031 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:01.126291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987398af-55ef-4cc6-a366-180d3120b031 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:01.126405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987398af-55ef-4cc6-a366-180d3120b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:01.215749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-987398af-55ef-4cc6-a366-180d3120b031 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:01.216986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-987398af-55ef-4cc6-a366-180d3120b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:01.323663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 663/1313] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:01.362199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f91286-f148-4e9d-b539-f6059b43f13f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:01.364034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b3f91286-f148-4e9d-b539-f6059b43f13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:01.471807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f91286-f148-4e9d-b539-f6059b43f13f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:01.545705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:01.548688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:01.549360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:01.549651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:01.549933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:01.552186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:01.663078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:01.664527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6365a-5d4b-4348-9519-6fcd17f6feb9 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:01.683828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 651/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:04.062815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b492b-42f4-44fa-b97c-15a95fe75df9 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:04.063381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b492b-42f4-44fa-b97c-15a95fe75df9 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:04.063381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b492b-42f4-44fa-b97c-15a95fe75df9 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:04.064667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 664/1315] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:04.070007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d7522-e511-432b-a614-2c9fe359e46e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:04.070458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d7522-e511-432b-a614-2c9fe359e46e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:04.070548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d7522-e511-432b-a614-2c9fe359e46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:04.139381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-685d7522-e511-432b-a614-2c9fe359e46e None None] MFA Rules not processed for user `e9d9ed1ce844447183749518f6265355`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:04.140615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-685d7522-e511-432b-a614-2c9fe359e46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:04.198732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:04.247284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 652/1316] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:04.290357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b492b-42f4-44fa-b97c-15a95fe75df9 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:04.292895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9f6b492b-42f4-44fa-b97c-15a95fe75df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:04.305927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:04.408642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b492b-42f4-44fa-b97c-15a95fe75df9 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:04.491929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:04.492583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:04.492861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:04.493144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:04.494843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:04.559500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:04.609373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:04.610860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87eec95-c400-40c5-b7b7-2155859ea40f service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:04.629266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 665/1317] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:07.225166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ce656-09af-4d83-b9bf-442c70ff430c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:07.225796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ce656-09af-4d83-b9bf-442c70ff430c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:07.225796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ce656-09af-4d83-b9bf-442c70ff430c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:07.227166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 653/1318] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:07.234164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da4dc2-8aa9-4d80-b204-7c4784bdf155 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:07.234998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da4dc2-8aa9-4d80-b204-7c4784bdf155 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:07.235149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da4dc2-8aa9-4d80-b204-7c4784bdf155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:07.311718 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a2da4dc2-8aa9-4d80-b204-7c4784bdf155 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:07.312938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a2da4dc2-8aa9-4d80-b204-7c4784bdf155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:07.411448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 666/1319] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:07.466594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed927c-03fb-466c-9c4a-5b9b9170e15d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:07.467001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed927c-03fb-466c-9c4a-5b9b9170e15d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:07.467398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed927c-03fb-466c-9c4a-5b9b9170e15d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:07.469540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 654/1320] 10.222.0.64 () {54 vars in 964 bytes} [Fri Jun 24 10:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:07.475500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec3313-ab85-415f-8baf-468ecd0c906d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:07.475723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec3313-ab85-415f-8baf-468ecd0c906d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:07.475997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec3313-ab85-415f-8baf-468ecd0c906d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:07.496622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:07.551929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2dec3313-ab85-415f-8baf-468ecd0c906d None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:07.553148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2dec3313-ab85-415f-8baf-468ecd0c906d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:07.634115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 667/1321] 10.222.0.64 () {58 vars in 1061 bytes} [Fri Jun 24 10:08:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:07.644156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed927c-03fb-466c-9c4a-5b9b9170e15d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:07.647331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1bed927c-03fb-466c-9c4a-5b9b9170e15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:07.767264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed927c-03fb-466c-9c4a-5b9b9170e15d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:07.846247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:07.846933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:07.847252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:07.847534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:07.849211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:07.857130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:07.982830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:07.984295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f2720-c0af-4c47-9027-6cce974282b5 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:08.004535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 655/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:11.185163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812c17-2caf-42fc-966b-01c3b0adc49b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:11.185749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812c17-2caf-42fc-966b-01c3b0adc49b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:11.185749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812c17-2caf-42fc-966b-01c3b0adc49b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:11.187162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 668/1323] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:11.192901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ea92b-707f-4c64-850d-d2df93c686d9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:11.193369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ea92b-707f-4c64-850d-d2df93c686d9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:11.193484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ea92b-707f-4c64-850d-d2df93c686d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:11.260703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5f4ea92b-707f-4c64-850d-d2df93c686d9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:11.261923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5f4ea92b-707f-4c64-850d-d2df93c686d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:11.326963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:11.370459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 656/1324] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:11.407378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812c17-2caf-42fc-966b-01c3b0adc49b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:11.409866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-91812c17-2caf-42fc-966b-01c3b0adc49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:11.531810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812c17-2caf-42fc-966b-01c3b0adc49b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:11.545113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:11.609668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:11.610326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:11.610610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:11.610920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:11.612577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:11.730499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:11.732200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18171ee-3a64-4d4f-b273-7f816ed49a3f service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:11.750680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 669/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:13.845658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ed60-caf2-4586-8dd2-66f227046874 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:13.846263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ed60-caf2-4586-8dd2-66f227046874 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:13.846263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ed60-caf2-4586-8dd2-66f227046874 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:13.847558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 657/1326] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:13.853300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c03215-0b5c-42fa-bcbc-23999d98cb6d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:13.854126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c03215-0b5c-42fa-bcbc-23999d98cb6d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:13.854248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c03215-0b5c-42fa-bcbc-23999d98cb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:13.904525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:13.924507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-26c03215-0b5c-42fa-bcbc-23999d98cb6d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:13.925857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-26c03215-0b5c-42fa-bcbc-23999d98cb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:14.028160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 670/1327] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:14.064037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749ed60-caf2-4586-8dd2-66f227046874 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:14.065700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5749ed60-caf2-4586-8dd2-66f227046874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:14.177191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749ed60-caf2-4586-8dd2-66f227046874 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:14.261129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:14.261820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:14.262105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:14.262375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:14.264083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:14.282980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:14.384522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:14.386018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c45310-d6f9-40bf-a94f-d167b902756b service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:14.406063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 658/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:16.490151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c731d9-c6d7-4173-a2fa-5efd7e5a530d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:16.490752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c731d9-c6d7-4173-a2fa-5efd7e5a530d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:16.490752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c731d9-c6d7-4173-a2fa-5efd7e5a530d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:16.492223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 671/1329] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:16.499659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679854-9a34-4e3f-a0e2-6b19656a5021 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:16.500185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679854-9a34-4e3f-a0e2-6b19656a5021 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:16.500285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679854-9a34-4e3f-a0e2-6b19656a5021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:16.572667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d5679854-9a34-4e3f-a0e2-6b19656a5021 None None] MFA Rules not processed for user `e9d9ed1ce844447183749518f6265355`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:16.573936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d5679854-9a34-4e3f-a0e2-6b19656a5021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:16.648225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:16.683614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 659/1330] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:16.721355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c731d9-c6d7-4173-a2fa-5efd7e5a530d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:16.723075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-99c731d9-c6d7-4173-a2fa-5efd7e5a530d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:16.838834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c731d9-c6d7-4173-a2fa-5efd7e5a530d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:16.861102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:16.918105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:16.918769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:16.919062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:16.919306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:16.920988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:17.052231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:17.053604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c46516-b595-4a62-90c3-f304ec05fc93 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:17.076065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 672/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:19.355472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41e1f8-2779-47bb-ad08-afa5908d88f7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:19.356060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41e1f8-2779-47bb-ad08-afa5908d88f7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:19.356060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41e1f8-2779-47bb-ad08-afa5908d88f7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:19.358508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 660/1332] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:19.364793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fedb31-d008-4f53-86ed-ce8f45f6d2be None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:19.365388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fedb31-d008-4f53-86ed-ce8f45f6d2be None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:19.365388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fedb31-d008-4f53-86ed-ce8f45f6d2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:19.436460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:19.440728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-84fedb31-d008-4f53-86ed-ce8f45f6d2be None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:19.443399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-84fedb31-d008-4f53-86ed-ce8f45f6d2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:19.546371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 673/1333] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:19.582634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41e1f8-2779-47bb-ad08-afa5908d88f7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:19.584213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9c41e1f8-2779-47bb-ad08-afa5908d88f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:19.695712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41e1f8-2779-47bb-ad08-afa5908d88f7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:19.780178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:19.780834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:19.781116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:19.781387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:19.783104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:19.811161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:19.898032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:19.899441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee332b-5f11-4722-8792-0c7fba199cfd service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:19.919169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 661/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:21.982506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e341f-bcfe-46ec-a960-574f97b02b74 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:21.983155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e341f-bcfe-46ec-a960-574f97b02b74 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:21.983155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e341f-bcfe-46ec-a960-574f97b02b74 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:21.984478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 674/1335] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:21.991384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd1f3-3966-4cca-8b44-f39b84b73b17 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:21.992142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd1f3-3966-4cca-8b44-f39b84b73b17 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:21.992293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd1f3-3966-4cca-8b44-f39b84b73b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:22.071507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-961fd1f3-3966-4cca-8b44-f39b84b73b17 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:22.072886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-961fd1f3-3966-4cca-8b44-f39b84b73b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:22.157195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:22.181437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 662/1336] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:22.225133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e341f-bcfe-46ec-a960-574f97b02b74 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:22.226696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6f4e341f-bcfe-46ec-a960-574f97b02b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:22.335960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e341f-bcfe-46ec-a960-574f97b02b74 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:22.366252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:22.419280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:22.419928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:22.420209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:22.420486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:22.422155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:22.555441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:22.556948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954ef71-81de-42ea-b3e3-acabafd20330 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:22.576202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 675/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:08:24.950082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99563e-b57b-4604-80a8-a221e4347bb5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:24.951000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99563e-b57b-4604-80a8-a221e4347bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:24.951000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99563e-b57b-4604-80a8-a221e4347bb5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:24.953234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 663/1338] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:24.959825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5f183-4452-4ead-a0e7-2a513c089995 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:24.960265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5f183-4452-4ead-a0e7-2a513c089995 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:24.960367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5f183-4452-4ead-a0e7-2a513c089995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:25.042141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-83f5f183-4452-4ead-a0e7-2a513c089995 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:25.044101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-83f5f183-4452-4ead-a0e7-2a513c089995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:25.055290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:25.154092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 676/1339] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:25.190198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99563e-b57b-4604-80a8-a221e4347bb5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:25.192103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ca99563e-b57b-4604-80a8-a221e4347bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:25.305653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99563e-b57b-4604-80a8-a221e4347bb5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:25.387359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:25.388000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:25.388287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:25.388546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:25.390217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:25.429548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:25.521058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:25.522528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc5a11-b4e5-437e-aa55-d5387bb95c3d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:25.542154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 664/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:29.628387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335e91-9af3-4b1e-8826-b9c0eff151c6 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:29.628936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335e91-9af3-4b1e-8826-b9c0eff151c6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:29.629850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335e91-9af3-4b1e-8826-b9c0eff151c6 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:29.631497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 677/1341] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:29.637496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4e89-98e7-4207-8cf8-a8bacf0de5cb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:29.638025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4e89-98e7-4207-8cf8-a8bacf0de5cb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:29.638025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4e89-98e7-4207-8cf8-a8bacf0de5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:29.714304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-348e4e89-98e7-4207-8cf8-a8bacf0de5cb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:29.715566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-348e4e89-98e7-4207-8cf8-a8bacf0de5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:29.811789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:29.825552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 665/1342] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:29.865224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335e91-9af3-4b1e-8826-b9c0eff151c6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:29.870271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a2335e91-9af3-4b1e-8826-b9c0eff151c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:29.977870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335e91-9af3-4b1e-8826-b9c0eff151c6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:29.981903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c5f5-1452-4bef-ae06-6335dd2c6442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:29.982567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c5f5-1452-4bef-ae06-6335dd2c6442 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:29.982891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c5f5-1452-4bef-ae06-6335dd2c6442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:29.983170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c5f5-1452-4bef-ae06-6335dd2c6442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:29.994131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45c5f5-1452-4bef-ae06-6335dd2c6442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:29.995626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45c5f5-1452-4bef-ae06-6335dd2c6442 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:30.005539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 678/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:32.709443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7c32-4cfb-4e83-b1de-7b949dc3726a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:32.709443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7c32-4cfb-4e83-b1de-7b949dc3726a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:32.709921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7c32-4cfb-4e83-b1de-7b949dc3726a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:32.711059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 666/1344] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:08:32.716967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1b10-612a-4f51-8404-017cc5220fe9 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:32.717537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1b10-612a-4f51-8404-017cc5220fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:32.717537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1b10-612a-4f51-8404-017cc5220fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:32.749815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-229b1b10-612a-4f51-8404-017cc5220fe9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:32.751095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-229b1b10-612a-4f51-8404-017cc5220fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:32.783034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:32.837052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 679/1345] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:32.875204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7c32-4cfb-4e83-b1de-7b949dc3726a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:32.886866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7c32-4cfb-4e83-b1de-7b949dc3726a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:32.890976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7792c-acd8-44b0-b6f2-11207f53436f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:32.891686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7792c-acd8-44b0-b6f2-11207f53436f service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:32.892034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7792c-acd8-44b0-b6f2-11207f53436f service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:32.892312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7792c-acd8-44b0-b6f2-11207f53436f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:32.901962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7792c-acd8-44b0-b6f2-11207f53436f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:32.903444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7792c-acd8-44b0-b6f2-11207f53436f service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:32.913200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 667/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:37.098510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413c05-c66c-42c0-aa98-0a5dcc3c520e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:37.099074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413c05-c66c-42c0-aa98-0a5dcc3c520e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:37.099195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413c05-c66c-42c0-aa98-0a5dcc3c520e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:37.100491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 680/1347] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:37.106458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fed44-938c-4b82-a07a-1633be834e11 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:37.106921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fed44-938c-4b82-a07a-1633be834e11 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:37.107301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fed44-938c-4b82-a07a-1633be834e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:37.126573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f21fed44-938c-4b82-a07a-1633be834e11 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:37.127830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f21fed44-938c-4b82-a07a-1633be834e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:37.136247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 668/1348] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:08:37.173992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413c05-c66c-42c0-aa98-0a5dcc3c520e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:37.176255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-59413c05-c66c-42c0-aa98-0a5dcc3c520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:37.189845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413c05-c66c-42c0-aa98-0a5dcc3c520e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:37.193699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b648e1b-e3cd-4faf-ab36-c17bd3e7a44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:37.194350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648e1b-e3cd-4faf-ab36-c17bd3e7a44c service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:37.194634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648e1b-e3cd-4faf-ab36-c17bd3e7a44c service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:37.194955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648e1b-e3cd-4faf-ab36-c17bd3e7a44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:37.204225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b648e1b-e3cd-4faf-ab36-c17bd3e7a44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:37.205676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b648e1b-e3cd-4faf-ab36-c17bd3e7a44c service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:37.215645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 681/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:42.246921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a1c20-0382-4860-84e1-66dfc899a4cd None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:42.247812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a1c20-0382-4860-84e1-66dfc899a4cd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:42.248298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a1c20-0382-4860-84e1-66dfc899a4cd None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:42.250221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 669/1350] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:42.255000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b0b56-f193-4fe7-b43d-f34889d5825c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:42.255420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b0b56-f193-4fe7-b43d-f34889d5825c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:42.255559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b0b56-f193-4fe7-b43d-f34889d5825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:42.362980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-772b0b56-f193-4fe7-b43d-f34889d5825c None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:42.364911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-772b0b56-f193-4fe7-b43d-f34889d5825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:42.439209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 682/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:46.667563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610e69-4a3c-4d5e-9076-25a4028c0ac1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:46.668208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610e69-4a3c-4d5e-9076-25a4028c0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:46.668208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610e69-4a3c-4d5e-9076-25a4028c0ac1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:46.669488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 670/1352] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:46.675139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875e59-0696-4fa3-97fc-bca039e82150 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:46.675644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875e59-0696-4fa3-97fc-bca039e82150 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:46.675799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875e59-0696-4fa3-97fc-bca039e82150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:46.698042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d7875e59-0696-4fa3-97fc-bca039e82150 None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:46.699500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d7875e59-0696-4fa3-97fc-bca039e82150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:46.710384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 683/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:08:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:47.121493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=13 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:48.417586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51ced4-8f24-4424-b8bf-e99af198578a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:48.418290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51ced4-8f24-4424-b8bf-e99af198578a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:48.418290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51ced4-8f24-4424-b8bf-e99af198578a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:48.419575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 671/1354] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:48.424784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab1b7-702a-4308-9b3e-5ad0fcc34a3a None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:48.425219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab1b7-702a-4308-9b3e-5ad0fcc34a3a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:48.425327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab1b7-702a-4308-9b3e-5ad0fcc34a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:48.444051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d63ab1b7-702a-4308-9b3e-5ad0fcc34a3a None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:48.445252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d63ab1b7-702a-4308-9b3e-5ad0fcc34a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:48.454183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 684/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:08:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:51.981774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb70fe1-8476-4f0c-abf1-549a8071c6ce None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:51.982344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb70fe1-8476-4f0c-abf1-549a8071c6ce None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:51.982344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb70fe1-8476-4f0c-abf1-549a8071c6ce None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:51.984812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 672/1356] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:51.990830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d3a24-89b7-4612-ade2-41537270064b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:51.991665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d3a24-89b7-4612-ade2-41537270064b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:51.992099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d3a24-89b7-4612-ade2-41537270064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:52.015609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b39d3a24-89b7-4612-ade2-41537270064b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:52.017947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b39d3a24-89b7-4612-ade2-41537270064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:52.029643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 685/1357] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:52.090520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4606-74c2-434c-b441-e57d45924e85 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:52.092119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4606-74c2-434c-b441-e57d45924e85 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:52.092773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4606-74c2-434c-b441-e57d45924e85 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:52.095462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 673/1358] 10.222.0.64 () {54 vars in 961 bytes} [Fri Jun 24 10:08:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:52.105737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50ab2-a59f-482c-a2c6-dc30ff2aea2c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:52.106771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50ab2-a59f-482c-a2c6-dc30ff2aea2c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:52.107397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc50ab2-a59f-482c-a2c6-dc30ff2aea2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:52.161104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:52.172203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3fc50ab2-a59f-482c-a2c6-dc30ff2aea2c None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:52.173496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3fc50ab2-a59f-482c-a2c6-dc30ff2aea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:52.281290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 686/1359] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Jun 24 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:08:52.290439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4606-74c2-434c-b441-e57d45924e85 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:52.306424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4606-74c2-434c-b441-e57d45924e85 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:52.312486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dfb48-6c76-4350-bf00-3f9e54ce099f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:52.313133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dfb48-6c76-4350-bf00-3f9e54ce099f service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:52.313416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dfb48-6c76-4350-bf00-3f9e54ce099f service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:52.313875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dfb48-6c76-4350-bf00-3f9e54ce099f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:52.323040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dfb48-6c76-4350-bf00-3f9e54ce099f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:52.324732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dfb48-6c76-4350-bf00-3f9e54ce099f service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:52.335207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 674/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:54.557530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a490e-ae87-418d-bbff-d8075f1a2ad2 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:54.558126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a490e-ae87-418d-bbff-d8075f1a2ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:54.558126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a490e-ae87-418d-bbff-d8075f1a2ad2 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:54.563551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 687/1361] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:54.564672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe830b2-008d-4f43-93f6-3e42ff95d905 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:54.565116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe830b2-008d-4f43-93f6-3e42ff95d905 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:54.565212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe830b2-008d-4f43-93f6-3e42ff95d905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:54.583715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bbe830b2-008d-4f43-93f6-3e42ff95d905 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:54.584938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bbe830b2-008d-4f43-93f6-3e42ff95d905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:54.593120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 675/1362] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:08:54.639887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a490e-ae87-418d-bbff-d8075f1a2ad2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:54.655610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a490e-ae87-418d-bbff-d8075f1a2ad2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:54.661129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d21ff0-0a5d-4811-a972-89ebdb3e4b4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:54.661781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d21ff0-0a5d-4811-a972-89ebdb3e4b4a service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:54.662066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d21ff0-0a5d-4811-a972-89ebdb3e4b4a service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:54.662345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d21ff0-0a5d-4811-a972-89ebdb3e4b4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:54.671007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d21ff0-0a5d-4811-a972-89ebdb3e4b4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:54.672624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d21ff0-0a5d-4811-a972-89ebdb3e4b4a service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:54.682444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 688/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:57.084448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761cb05-e441-4e88-ac07-f3ba5af8d11b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:57.084448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761cb05-e441-4e88-ac07-f3ba5af8d11b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:57.084448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761cb05-e441-4e88-ac07-f3ba5af8d11b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:57.085977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 676/1364] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:57.091902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7ecf-c540-4509-8e7a-87184835099f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:57.092731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7ecf-c540-4509-8e7a-87184835099f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:57.093178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7ecf-c540-4509-8e7a-87184835099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:57.115418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d56b7ecf-c540-4509-8e7a-87184835099f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:57.116807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d56b7ecf-c540-4509-8e7a-87184835099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:57.124828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 689/1365] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:57.171448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890dba3-1406-45ef-a806-2a7931025b2d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:57.171730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890dba3-1406-45ef-a806-2a7931025b2d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:57.172002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890dba3-1406-45ef-a806-2a7931025b2d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:57.173424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 677/1366] 10.222.0.64 () {54 vars in 961 bytes} [Fri Jun 24 10:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:57.179471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604835-2145-4e58-ba28-c7d4542f5dc0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:57.179728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604835-2145-4e58-ba28-c7d4542f5dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:57.180029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604835-2145-4e58-ba28-c7d4542f5dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:57.199097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bb604835-2145-4e58-ba28-c7d4542f5dc0 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:57.200299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bb604835-2145-4e58-ba28-c7d4542f5dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:57.210897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 690/1367] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Jun 24 10:08:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:08:57.218947 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890dba3-1406-45ef-a806-2a7931025b2d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:57.234092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890dba3-1406-45ef-a806-2a7931025b2d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:57.240026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07598e74-8d56-4eed-be15-7832f544af4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:57.240751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598e74-8d56-4eed-be15-7832f544af4c service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:57.241036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598e74-8d56-4eed-be15-7832f544af4c service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:57.241312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598e74-8d56-4eed-be15-7832f544af4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:57.249733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07598e74-8d56-4eed-be15-7832f544af4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:57.251378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07598e74-8d56-4eed-be15-7832f544af4c service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:57.261173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 678/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:59.405777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180ae4-4943-42fe-a55d-508e0dcab2c1 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:59.406365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180ae4-4943-42fe-a55d-508e0dcab2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:59.406365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180ae4-4943-42fe-a55d-508e0dcab2c1 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:59.407748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 691/1369] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:59.414801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985678f3-558a-4aa2-82b3-3817eca914b9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:59.415391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985678f3-558a-4aa2-82b3-3817eca914b9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:59.415391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985678f3-558a-4aa2-82b3-3817eca914b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:59.435390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-985678f3-558a-4aa2-82b3-3817eca914b9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:59.436587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-985678f3-558a-4aa2-82b3-3817eca914b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:59.445659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 679/1370] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:59.490900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180ae4-4943-42fe-a55d-508e0dcab2c1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:59.505954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180ae4-4943-42fe-a55d-508e0dcab2c1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:59.511603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d799a-858a-48b2-8bf7-32a4317564bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:59.512239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d799a-858a-48b2-8bf7-32a4317564bf service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:59.512517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d799a-858a-48b2-8bf7-32a4317564bf service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:59.512777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d799a-858a-48b2-8bf7-32a4317564bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:59.522023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d799a-858a-48b2-8bf7-32a4317564bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:59.523646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d799a-858a-48b2-8bf7-32a4317564bf service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:59.533515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 692/1371] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:00.172040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be24bf-fa3a-49bf-8a28-150ea296e460 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:00.172279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be24bf-fa3a-49bf-8a28-150ea296e460 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:00.172558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be24bf-fa3a-49bf-8a28-150ea296e460 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:00.174125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 680/1372] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:00.179805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22eed54-d53a-45dc-87fe-1f5aa437ab37 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:00.180186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22eed54-d53a-45dc-87fe-1f5aa437ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:00.180552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22eed54-d53a-45dc-87fe-1f5aa437ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:00.199458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a22eed54-d53a-45dc-87fe-1f5aa437ab37 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:00.200741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a22eed54-d53a-45dc-87fe-1f5aa437ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:00.209685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 693/1373] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:00.594460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b18c2a-c998-4523-bfbe-d816f1277ae9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:00.595460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b18c2a-c998-4523-bfbe-d816f1277ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:00.595966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b18c2a-c998-4523-bfbe-d816f1277ae9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:00.598005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 681/1374] 10.222.0.64 () {54 vars in 963 bytes} [Fri Jun 24 10:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:00.604802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265637e5-06c8-4b5e-a059-3bae875dc633 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:00.605270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265637e5-06c8-4b5e-a059-3bae875dc633 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:00.605371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265637e5-06c8-4b5e-a059-3bae875dc633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:00.667560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-265637e5-06c8-4b5e-a059-3bae875dc633 None None] MFA Rules not processed for user `dfd4560d3f144b339adb62b9ddf118ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:00.670453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-265637e5-06c8-4b5e-a059-3bae875dc633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:00.759004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 694/1375] 10.222.0.64 () {58 vars in 1060 bytes} [Fri Jun 24 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:00.767059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b18c2a-c998-4523-bfbe-d816f1277ae9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:00.778524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b18c2a-c998-4523-bfbe-d816f1277ae9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:00.782543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006058b-8100-4812-a634-bb21cf9e7444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:00.783460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006058b-8100-4812-a634-bb21cf9e7444 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:00.783960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006058b-8100-4812-a634-bb21cf9e7444 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:00.784434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006058b-8100-4812-a634-bb21cf9e7444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:00.795255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006058b-8100-4812-a634-bb21cf9e7444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:00.797175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006058b-8100-4812-a634-bb21cf9e7444 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:00.808985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 682/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:01.693181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6b1e4-678a-48a3-aafe-3c36f44a943b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:01.693712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6b1e4-678a-48a3-aafe-3c36f44a943b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:01.693712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6b1e4-678a-48a3-aafe-3c36f44a943b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:01.695035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 695/1377] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:01.700536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705270cf-fcff-47e7-a833-4a36863ed2a7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:01.700975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705270cf-fcff-47e7-a833-4a36863ed2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:01.701210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705270cf-fcff-47e7-a833-4a36863ed2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:01.721105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-705270cf-fcff-47e7-a833-4a36863ed2a7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:01.722838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-705270cf-fcff-47e7-a833-4a36863ed2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:01.732220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 683/1378] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:01.776393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6b1e4-678a-48a3-aafe-3c36f44a943b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:01.791994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6b1e4-678a-48a3-aafe-3c36f44a943b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:01.797772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c5ed7-ff21-443c-a821-7eba6facd3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:01.798413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c5ed7-ff21-443c-a821-7eba6facd3fe service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:01.798694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c5ed7-ff21-443c-a821-7eba6facd3fe service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:01.798996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c5ed7-ff21-443c-a821-7eba6facd3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:01.807715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c5ed7-ff21-443c-a821-7eba6facd3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:01.809308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c5ed7-ff21-443c-a821-7eba6facd3fe service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:01.814044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:09:01.820772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 696/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:03.871370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964d0f-9797-45d0-9521-db36dc468eef None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:03.871943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964d0f-9797-45d0-9521-db36dc468eef None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:03.871943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964d0f-9797-45d0-9521-db36dc468eef None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:03.873242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 684/1380] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:09:03.878735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0d647-31dc-41f9-853d-b505a9a59b5d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:03.879254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0d647-31dc-41f9-853d-b505a9a59b5d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:03.879368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0d647-31dc-41f9-853d-b505a9a59b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:03.899034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-eda0d647-31dc-41f9-853d-b505a9a59b5d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:03.900244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-eda0d647-31dc-41f9-853d-b505a9a59b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:03.909910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 697/1381] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:03.955671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87964d0f-9797-45d0-9521-db36dc468eef None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:03.970446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87964d0f-9797-45d0-9521-db36dc468eef None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:03.976022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ba59e-4023-43a9-ba94-b1adb1a63204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:03.976658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ba59e-4023-43a9-ba94-b1adb1a63204 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:03.976939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ba59e-4023-43a9-ba94-b1adb1a63204 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:03.977212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ba59e-4023-43a9-ba94-b1adb1a63204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:03.985747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ba59e-4023-43a9-ba94-b1adb1a63204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:03.987390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ba59e-4023-43a9-ba94-b1adb1a63204 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:03.997367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 685/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:06.075580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648efdf-cf7a-4ba8-92fa-ad227e43ab7c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:06.076211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648efdf-cf7a-4ba8-92fa-ad227e43ab7c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:06.076211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648efdf-cf7a-4ba8-92fa-ad227e43ab7c None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:06.078814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 698/1383] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:06.082740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02930ab-0e43-464d-9b07-f9b637705a99 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:06.083179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02930ab-0e43-464d-9b07-f9b637705a99 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:06.083299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02930ab-0e43-464d-9b07-f9b637705a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:06.101568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f02930ab-0e43-464d-9b07-f9b637705a99 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:06.102764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f02930ab-0e43-464d-9b07-f9b637705a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:06.112015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 686/1384] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:06.154384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648efdf-cf7a-4ba8-92fa-ad227e43ab7c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:06.173277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648efdf-cf7a-4ba8-92fa-ad227e43ab7c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:06.178844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece853e6-86f5-47f9-a311-0cf5f0ec8188 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:06.179506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece853e6-86f5-47f9-a311-0cf5f0ec8188 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:06.179791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece853e6-86f5-47f9-a311-0cf5f0ec8188 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:06.180070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece853e6-86f5-47f9-a311-0cf5f0ec8188 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:06.189057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece853e6-86f5-47f9-a311-0cf5f0ec8188 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:06.190663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece853e6-86f5-47f9-a311-0cf5f0ec8188 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:06.201027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 699/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:08.253024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aea75-c67b-4ddc-87de-d6783a2f95e1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:08.253611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aea75-c67b-4ddc-87de-d6783a2f95e1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:08.253611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aea75-c67b-4ddc-87de-d6783a2f95e1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:08.256155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 687/1386] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:08.261307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550eab-a9ae-4def-941b-4389b8fcf9a8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:08.261792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550eab-a9ae-4def-941b-4389b8fcf9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:08.261885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550eab-a9ae-4def-941b-4389b8fcf9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:08.280549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-78550eab-a9ae-4def-941b-4389b8fcf9a8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:08.281665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-78550eab-a9ae-4def-941b-4389b8fcf9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:08.290098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 700/1387] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:08.335890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aea75-c67b-4ddc-87de-d6783a2f95e1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:08.352495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aea75-c67b-4ddc-87de-d6783a2f95e1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:08.358315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c48892-caf5-4534-a386-fd53ee217c26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:08.358994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c48892-caf5-4534-a386-fd53ee217c26 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:08.359271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c48892-caf5-4534-a386-fd53ee217c26 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:08.359540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c48892-caf5-4534-a386-fd53ee217c26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:08.368821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c48892-caf5-4534-a386-fd53ee217c26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:08.370418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c48892-caf5-4534-a386-fd53ee217c26 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:08.381786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 688/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:10.581594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39052-da8a-4fba-b20e-ecd77a2160dd None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:10.582207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39052-da8a-4fba-b20e-ecd77a2160dd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:10.582207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39052-da8a-4fba-b20e-ecd77a2160dd None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:10.583673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 701/1389] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:10.589951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db728b70-8691-4f76-a67a-41fa6182719b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:10.590451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db728b70-8691-4f76-a67a-41fa6182719b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:10.590750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db728b70-8691-4f76-a67a-41fa6182719b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:10.611707 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-db728b70-8691-4f76-a67a-41fa6182719b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:10.613069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-db728b70-8691-4f76-a67a-41fa6182719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:10.621876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 689/1390] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:10.672290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39052-da8a-4fba-b20e-ecd77a2160dd None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:10.690629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39052-da8a-4fba-b20e-ecd77a2160dd None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:10.696975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28003576-77dc-4884-9f50-7aae4dc29d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:10.697661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28003576-77dc-4884-9f50-7aae4dc29d48 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:10.697982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28003576-77dc-4884-9f50-7aae4dc29d48 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:10.698253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28003576-77dc-4884-9f50-7aae4dc29d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:10.707774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28003576-77dc-4884-9f50-7aae4dc29d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:10.709476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28003576-77dc-4884-9f50-7aae4dc29d48 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:10.720992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 702/1391] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:13.296151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589853e-4810-46ed-9b05-1feb43ee0f86 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:13.296997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589853e-4810-46ed-9b05-1feb43ee0f86 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:13.296997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589853e-4810-46ed-9b05-1feb43ee0f86 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:13.298446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 690/1392] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:13.304712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11818788-6f39-4769-ad63-411ee3412f14 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:13.305299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11818788-6f39-4769-ad63-411ee3412f14 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:13.305393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11818788-6f39-4769-ad63-411ee3412f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:13.327580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-11818788-6f39-4769-ad63-411ee3412f14 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:13.328875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-11818788-6f39-4769-ad63-411ee3412f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:13.338244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 703/1393] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:09:13.386480 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589853e-4810-46ed-9b05-1feb43ee0f86 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:13.402509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589853e-4810-46ed-9b05-1feb43ee0f86 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:13.408153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f47c10-e49d-462f-9f79-37ac761b8865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:13.408821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f47c10-e49d-462f-9f79-37ac761b8865 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:13.409111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f47c10-e49d-462f-9f79-37ac761b8865 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:13.409410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f47c10-e49d-462f-9f79-37ac761b8865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:13.418830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f47c10-e49d-462f-9f79-37ac761b8865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:13.420665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f47c10-e49d-462f-9f79-37ac761b8865 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:13.431096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 691/1394] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:15.500794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c20f0-4cce-4d16-a4f8-fd14530a3418 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:15.501455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c20f0-4cce-4d16-a4f8-fd14530a3418 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:15.501455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c20f0-4cce-4d16-a4f8-fd14530a3418 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:15.502747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 704/1395] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:15.508068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812952-46d5-4dc5-8387-7a5de2c0fdab None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:15.508558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812952-46d5-4dc5-8387-7a5de2c0fdab None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:15.508664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812952-46d5-4dc5-8387-7a5de2c0fdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:15.527932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-df812952-46d5-4dc5-8387-7a5de2c0fdab None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:15.529151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-df812952-46d5-4dc5-8387-7a5de2c0fdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:15.537779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 692/1396] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:15.581772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c20f0-4cce-4d16-a4f8-fd14530a3418 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:15.597695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c20f0-4cce-4d16-a4f8-fd14530a3418 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:15.605883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f322d-aabb-4d25-b8f6-37cddce50252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:15.606559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f322d-aabb-4d25-b8f6-37cddce50252 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:15.606858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f322d-aabb-4d25-b8f6-37cddce50252 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:15.607136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f322d-aabb-4d25-b8f6-37cddce50252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:15.616056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f322d-aabb-4d25-b8f6-37cddce50252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:15.617652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f322d-aabb-4d25-b8f6-37cddce50252 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:15.627516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 705/1397] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:15.982582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265286f-5491-4f11-9efa-1634f613c51e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:15.982886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265286f-5491-4f11-9efa-1634f613c51e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:15.983160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265286f-5491-4f11-9efa-1634f613c51e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:15.984569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 693/1398] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:15.990302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cae6d-b66f-474c-912d-c61abce38149 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:15.990669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cae6d-b66f-474c-912d-c61abce38149 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:15.991049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cae6d-b66f-474c-912d-c61abce38149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:16.009926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d52cae6d-b66f-474c-912d-c61abce38149 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:16.011178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d52cae6d-b66f-474c-912d-c61abce38149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:16.020239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 706/1399] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:16.075950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcd952-5faa-4932-9375-d073161e5be3 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:16.076560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcd952-5faa-4932-9375-d073161e5be3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:16.076841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcd952-5faa-4932-9375-d073161e5be3 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:16.078311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 694/1400] 10.222.0.64 () {54 vars in 962 bytes} [Fri Jun 24 10:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:16.083807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f584c1f-6656-482f-8210-0d713ac6248c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:16.084104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f584c1f-6656-482f-8210-0d713ac6248c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:16.084384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f584c1f-6656-482f-8210-0d713ac6248c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:16.137043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2f584c1f-6656-482f-8210-0d713ac6248c None None] MFA Rules not processed for user `bb0cbf8b93e741c2b500cbf1cb1ad5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:16.138264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2f584c1f-6656-482f-8210-0d713ac6248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:16.201903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 707/1401] 10.222.0.64 () {58 vars in 1059 bytes} [Fri Jun 24 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:16.209779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcd952-5faa-4932-9375-d073161e5be3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:16.221028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcd952-5faa-4932-9375-d073161e5be3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:16.224819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399456d-d20f-40bd-a2a8-7dbd054acea3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:16.225486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399456d-d20f-40bd-a2a8-7dbd054acea3 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:16.225762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399456d-d20f-40bd-a2a8-7dbd054acea3 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:16.226033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399456d-d20f-40bd-a2a8-7dbd054acea3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:16.236982 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399456d-d20f-40bd-a2a8-7dbd054acea3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:16.238718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399456d-d20f-40bd-a2a8-7dbd054acea3 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:16.248405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 695/1402] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:09:16.988354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd831-f94f-4eee-8473-9f10dd083e2c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:16.988930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd831-f94f-4eee-8473-9f10dd083e2c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:16.989284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fd831-f94f-4eee-8473-9f10dd083e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.038031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-631fd831-f94f-4eee-8473-9f10dd083e2c None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:17.039268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-631fd831-f94f-4eee-8473-9f10dd083e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:17.079304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:09:17.103730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 708/1403] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:17.148623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca71e-3dae-42d2-91f3-7ea0bacac8f8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:17.149148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca71e-3dae-42d2-91f3-7ea0bacac8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:17.149148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca71e-3dae-42d2-91f3-7ea0bacac8f8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.150620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 696/1404] 10.222.0.64 () {54 vars in 942 bytes} [Fri Jun 24 10:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:17.157270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3a93-0b4b-4eee-b1a1-852f13ed2a21 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:17.157589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3a93-0b4b-4eee-b1a1-852f13ed2a21 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:17.157865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3a93-0b4b-4eee-b1a1-852f13ed2a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.209918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5eaa3a93-0b4b-4eee-b1a1-852f13ed2a21 None None] MFA Rules not processed for user `19fd34c5b9e34440b7c3752ff1cdf4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:17.211076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5eaa3a93-0b4b-4eee-b1a1-852f13ed2a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:17.286625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 709/1405] 10.222.0.64 () {58 vars in 1039 bytes} [Fri Jun 24 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:17.295965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aca71e-3dae-42d2-91f3-7ea0bacac8f8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:17.304002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=61 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:09:17.311477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aca71e-3dae-42d2-91f3-7ea0bacac8f8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:17.317185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ac363-21c5-4dd3-81ec-ae805fd5d210 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:17.317827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ac363-21c5-4dd3-81ec-ae805fd5d210 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:17.318107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ac363-21c5-4dd3-81ec-ae805fd5d210 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:17.318378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ac363-21c5-4dd3-81ec-ae805fd5d210 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.326849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ac363-21c5-4dd3-81ec-ae805fd5d210 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:17.328452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ac363-21c5-4dd3-81ec-ae805fd5d210 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:17.336971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 697/1406] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:09:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:09:17.701423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b16287-b175-4cf0-b89a-5b574c9a7f9d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:17.701712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b16287-b175-4cf0-b89a-5b574c9a7f9d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:17.701994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b16287-b175-4cf0-b89a-5b574c9a7f9d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.707934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 710/1407] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:09:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:17.709345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebb8d5-9e9a-4e72-9ee8-a8ad3b6aa5d7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:17.709639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebb8d5-9e9a-4e72-9ee8-a8ad3b6aa5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:17.709918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebb8d5-9e9a-4e72-9ee8-a8ad3b6aa5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.742274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cdebb8d5-9e9a-4e72-9ee8-a8ad3b6aa5d7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:17.743589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cdebb8d5-9e9a-4e72-9ee8-a8ad3b6aa5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:17.752794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 698/1408] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:09:17.799187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b16287-b175-4cf0-b89a-5b574c9a7f9d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:17.815912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b16287-b175-4cf0-b89a-5b574c9a7f9d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:17.821568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6db241-ee9a-400a-a270-d2602261736f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:17.822298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6db241-ee9a-400a-a270-d2602261736f service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:17.822647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6db241-ee9a-400a-a270-d2602261736f service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:17.823055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6db241-ee9a-400a-a270-d2602261736f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:17.833785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6db241-ee9a-400a-a270-d2602261736f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:17.835447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6db241-ee9a-400a-a270-d2602261736f service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:17.845578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 711/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:27.731257 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=8 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:09:27.841047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=59 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:09:39.395866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e662eee-b80a-4e67-9d24-83bc5a97ce3e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:39.396478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e662eee-b80a-4e67-9d24-83bc5a97ce3e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:39.396478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e662eee-b80a-4e67-9d24-83bc5a97ce3e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:39.397863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 699/1410] 10.222.0.64 () {54 vars in 918 bytes} [Fri Jun 24 10:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:39.403395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c4260-50f0-4a6a-aced-09d383e0cce8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:39.403848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c4260-50f0-4a6a-aced-09d383e0cce8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:39.403951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c4260-50f0-4a6a-aced-09d383e0cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:39.426164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-046c4260-50f0-4a6a-aced-09d383e0cce8 None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:39.427999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-046c4260-50f0-4a6a-aced-09d383e0cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:39.442146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 712/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Jun 24 10:09:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:39.456566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776b245-e151-4594-8fda-f17b0dd9ddd8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:39.456885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776b245-e151-4594-8fda-f17b0dd9ddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:39.457167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776b245-e151-4594-8fda-f17b0dd9ddd8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:39.458639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 700/1412] 10.222.0.64 () {54 vars in 918 bytes} [Fri Jun 24 10:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:39.463465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558d8f8-0fe2-4ec4-a381-621ce4e0e6dd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:39.463760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558d8f8-0fe2-4ec4-a381-621ce4e0e6dd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:39.464034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558d8f8-0fe2-4ec4-a381-621ce4e0e6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:39.483987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0558d8f8-0fe2-4ec4-a381-621ce4e0e6dd None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:39.485180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0558d8f8-0fe2-4ec4-a381-621ce4e0e6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:39.495896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 713/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Jun 24 10:09:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:41.986532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647e1bc-3212-4a20-9ed0-4952635eee20 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:41.987168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647e1bc-3212-4a20-9ed0-4952635eee20 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:41.987168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647e1bc-3212-4a20-9ed0-4952635eee20 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:41.988618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:41.994152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad6042-17bf-4b16-880c-6d7e116a05f9 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:41.994595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad6042-17bf-4b16-880c-6d7e116a05f9 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:41.994727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad6042-17bf-4b16-880c-6d7e116a05f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:42.015257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3bad6042-17bf-4b16-880c-6d7e116a05f9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:42.016470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3bad6042-17bf-4b16-880c-6d7e116a05f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:42.025932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:42.073526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647e1bc-3212-4a20-9ed0-4952635eee20 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:42.089214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647e1bc-3212-4a20-9ed0-4952635eee20 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:42.094798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8dd6a-10d2-4e2a-b0c8-7c629d1a8d8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:42.095439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8dd6a-10d2-4e2a-b0c8-7c629d1a8d8e service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:42.095718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8dd6a-10d2-4e2a-b0c8-7c629d1a8d8e service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:42.095993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8dd6a-10d2-4e2a-b0c8-7c629d1a8d8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:42.104854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8dd6a-10d2-4e2a-b0c8-7c629d1a8d8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:42.106461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8dd6a-10d2-4e2a-b0c8-7c629d1a8d8e service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:42.116372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 702/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:43.034066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0c7c-197e-4904-a648-d0e41d7e6314 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:43.034763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0c7c-197e-4904-a648-d0e41d7e6314 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:43.034763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0c7c-197e-4904-a648-d0e41d7e6314 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:43.036243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:43.043491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945a0a1-7363-4f5a-bee1-09911141caf1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:43.043961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945a0a1-7363-4f5a-bee1-09911141caf1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:43.044069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945a0a1-7363-4f5a-bee1-09911141caf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:43.066837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0945a0a1-7363-4f5a-bee1-09911141caf1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:43.068071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0945a0a1-7363-4f5a-bee1-09911141caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:43.078191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:43.234992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0c7c-197e-4904-a648-d0e41d7e6314 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:43.258339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0c7c-197e-4904-a648-d0e41d7e6314 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:43.265164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4175e-91cf-4bb9-9c52-280c7f24bad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:43.265834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4175e-91cf-4bb9-9c52-280c7f24bad0 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:43.266091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4175e-91cf-4bb9-9c52-280c7f24bad0 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:43.266367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4175e-91cf-4bb9-9c52-280c7f24bad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:43.275518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4175e-91cf-4bb9-9c52-280c7f24bad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:43.277113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4175e-91cf-4bb9-9c52-280c7f24bad0 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:43.288290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 716/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:44.168656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea04063-cfb5-461b-9e18-b9c5c7ef0244 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:44.169589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea04063-cfb5-461b-9e18-b9c5c7ef0244 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:44.169589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea04063-cfb5-461b-9e18-b9c5c7ef0244 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:44.171833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:44.178414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93178b31-0607-42f6-b74d-2d44234ae798 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:44.178886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93178b31-0607-42f6-b74d-2d44234ae798 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:44.178998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93178b31-0607-42f6-b74d-2d44234ae798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:44.197406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-93178b31-0607-42f6-b74d-2d44234ae798 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:44.198609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-93178b31-0607-42f6-b74d-2d44234ae798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:44.207656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:44.361219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea04063-cfb5-461b-9e18-b9c5c7ef0244 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:44.382447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea04063-cfb5-461b-9e18-b9c5c7ef0244 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:44.391323 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8f830-cf12-4a1d-9d8f-798e3d1ed788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:44.392303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8f830-cf12-4a1d-9d8f-798e3d1ed788 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:44.392728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8f830-cf12-4a1d-9d8f-798e3d1ed788 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:44.393149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8f830-cf12-4a1d-9d8f-798e3d1ed788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:44.402677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8f830-cf12-4a1d-9d8f-798e3d1ed788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:44.404317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8f830-cf12-4a1d-9d8f-798e3d1ed788 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:44.414531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 705/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:45.277608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35998b6-cd6d-4ff0-8e17-5f0a5f523e1c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:45.278503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35998b6-cd6d-4ff0-8e17-5f0a5f523e1c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:45.278503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35998b6-cd6d-4ff0-8e17-5f0a5f523e1c None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:45.280704 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:45.286360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e637-c07a-40e0-901b-142b16db20db None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:45.286869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e637-c07a-40e0-901b-142b16db20db None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:45.286966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e637-c07a-40e0-901b-142b16db20db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:45.307124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-17a7e637-c07a-40e0-901b-142b16db20db None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:45.308372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-17a7e637-c07a-40e0-901b-142b16db20db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:45.316845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:09:45.481561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35998b6-cd6d-4ff0-8e17-5f0a5f523e1c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:45.497739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35998b6-cd6d-4ff0-8e17-5f0a5f523e1c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:45.505677 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2434b0-4f3d-498c-98d4-bb2e46148e61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:45.506341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2434b0-4f3d-498c-98d4-bb2e46148e61 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:45.506615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2434b0-4f3d-498c-98d4-bb2e46148e61 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:45.506914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2434b0-4f3d-498c-98d4-bb2e46148e61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:45.516608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2434b0-4f3d-498c-98d4-bb2e46148e61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:45.518611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2434b0-4f3d-498c-98d4-bb2e46148e61 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:45.529019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 719/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:46.485817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17cbb-596c-489a-a3d8-eba97ffe080b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:46.486683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17cbb-596c-489a-a3d8-eba97ffe080b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:46.486917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17cbb-596c-489a-a3d8-eba97ffe080b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:46.489485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:46.497694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64de3b-5b5d-45e3-a6b7-47cb54134219 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:46.498148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64de3b-5b5d-45e3-a6b7-47cb54134219 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:46.498269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64de3b-5b5d-45e3-a6b7-47cb54134219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:46.523627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-bd64de3b-5b5d-45e3-a6b7-47cb54134219 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:46.525344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-bd64de3b-5b5d-45e3-a6b7-47cb54134219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:46.539824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:46.693756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d17cbb-596c-489a-a3d8-eba97ffe080b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:46.709801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d17cbb-596c-489a-a3d8-eba97ffe080b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:46.715610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe8c55-a8b4-4fd3-863a-347d4f76754c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:46.716225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8c55-a8b4-4fd3-863a-347d4f76754c service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:46.716538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8c55-a8b4-4fd3-863a-347d4f76754c service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:46.716813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8c55-a8b4-4fd3-863a-347d4f76754c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:46.725692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe8c55-a8b4-4fd3-863a-347d4f76754c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:46.727701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe8c55-a8b4-4fd3-863a-347d4f76754c service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:46.737979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 708/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:47.573476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be4ade-14d8-4d7b-b32f-9ce5c4e6b40e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:47.574043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be4ade-14d8-4d7b-b32f-9ce5c4e6b40e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:47.574043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be4ade-14d8-4d7b-b32f-9ce5c4e6b40e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:47.575469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:47.581354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275370f4-6160-4f6e-a696-7699be73688d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:47.581820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275370f4-6160-4f6e-a696-7699be73688d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:47.581917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275370f4-6160-4f6e-a696-7699be73688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:47.606405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-275370f4-6160-4f6e-a696-7699be73688d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:47.607704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-275370f4-6160-4f6e-a696-7699be73688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:47.617212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:47.778049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be4ade-14d8-4d7b-b32f-9ce5c4e6b40e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:47.793717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be4ade-14d8-4d7b-b32f-9ce5c4e6b40e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:47.801238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ab4e7-29f8-4e47-a683-be1f2d200b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:47.801897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab4e7-29f8-4e47-a683-be1f2d200b53 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:47.802171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab4e7-29f8-4e47-a683-be1f2d200b53 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:47.802438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab4e7-29f8-4e47-a683-be1f2d200b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:47.812782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ab4e7-29f8-4e47-a683-be1f2d200b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:47.814381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ab4e7-29f8-4e47-a683-be1f2d200b53 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:47.825651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 722/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:48.627948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e117c-955d-403c-99ff-a281400635aa None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:48.628878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e117c-955d-403c-99ff-a281400635aa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:48.628878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e117c-955d-403c-99ff-a281400635aa None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:48.630983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:48.639259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa48173-acc2-4b87-a110-d02de4b0f55f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:48.639713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa48173-acc2-4b87-a110-d02de4b0f55f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:48.639848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa48173-acc2-4b87-a110-d02de4b0f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:48.661572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-efa48173-acc2-4b87-a110-d02de4b0f55f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:48.662845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-efa48173-acc2-4b87-a110-d02de4b0f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:48.672053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:48.825116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e117c-955d-403c-99ff-a281400635aa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:48.844069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e117c-955d-403c-99ff-a281400635aa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:48.850495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fdf31-79f1-447c-b36e-f744d4c9d423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:48.851161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fdf31-79f1-447c-b36e-f744d4c9d423 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:48.851806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fdf31-79f1-447c-b36e-f744d4c9d423 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:48.852079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fdf31-79f1-447c-b36e-f744d4c9d423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:48.861122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fdf31-79f1-447c-b36e-f744d4c9d423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:48.863001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fdf31-79f1-447c-b36e-f744d4c9d423 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:48.873195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 711/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:49.684074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be08b2-0f4a-42c4-8656-56217b501fd3 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:49.684653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be08b2-0f4a-42c4-8656-56217b501fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:49.684653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be08b2-0f4a-42c4-8656-56217b501fd3 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:49.686038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:49.691524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be19834-3e81-48e3-a928-62a03ba2aea1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:49.691978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be19834-3e81-48e3-a928-62a03ba2aea1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:49.692091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be19834-3e81-48e3-a928-62a03ba2aea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:49.711515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8be19834-3e81-48e3-a928-62a03ba2aea1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:49.712724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8be19834-3e81-48e3-a928-62a03ba2aea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:49.721232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:09:49.874197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be08b2-0f4a-42c4-8656-56217b501fd3 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:49.890328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be08b2-0f4a-42c4-8656-56217b501fd3 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:49.896882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfb0f-7437-40db-89e6-5a8f61810f59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:49.897523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfb0f-7437-40db-89e6-5a8f61810f59 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:49.897809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfb0f-7437-40db-89e6-5a8f61810f59 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:49.898077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfb0f-7437-40db-89e6-5a8f61810f59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:49.906655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfb0f-7437-40db-89e6-5a8f61810f59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:49.908296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfb0f-7437-40db-89e6-5a8f61810f59 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:49.917912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 725/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:50.718877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6e7f2-320b-4f3e-bd21-3280c654309f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:50.719441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6e7f2-320b-4f3e-bd21-3280c654309f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:50.719441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6e7f2-320b-4f3e-bd21-3280c654309f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:50.720834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:50.726684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d4fe-f4df-49f8-82e7-c32ecf5ccbc8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:50.727168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d4fe-f4df-49f8-82e7-c32ecf5ccbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:50.727281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d4fe-f4df-49f8-82e7-c32ecf5ccbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:50.746016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b380d4fe-f4df-49f8-82e7-c32ecf5ccbc8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:50.747249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b380d4fe-f4df-49f8-82e7-c32ecf5ccbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:50.756070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:50.903133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6e7f2-320b-4f3e-bd21-3280c654309f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:50.927065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6e7f2-320b-4f3e-bd21-3280c654309f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:50.934041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a17a81-8d94-46de-b2e9-def2707850d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:50.934679 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a17a81-8d94-46de-b2e9-def2707850d0 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:50.934994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a17a81-8d94-46de-b2e9-def2707850d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:50.935268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a17a81-8d94-46de-b2e9-def2707850d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:50.943717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a17a81-8d94-46de-b2e9-def2707850d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:50.945307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a17a81-8d94-46de-b2e9-def2707850d0 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:50.955072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 714/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:52.911452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fca318-737e-4caa-b094-78b8aadf5d63 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:52.912364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fca318-737e-4caa-b094-78b8aadf5d63 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:52.912364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fca318-737e-4caa-b094-78b8aadf5d63 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:52.914490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:09:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:52.923423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c24f5-b045-43a1-8f5a-78480cbad2a7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:52.923866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c24f5-b045-43a1-8f5a-78480cbad2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:52.924040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c24f5-b045-43a1-8f5a-78480cbad2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:52.949855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ad8c24f5-b045-43a1-8f5a-78480cbad2a7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:52.951100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad8c24f5-b045-43a1-8f5a-78480cbad2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:52.960798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:53.003806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fca318-737e-4caa-b094-78b8aadf5d63 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:53.017030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fca318-737e-4caa-b094-78b8aadf5d63 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:53.022757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af9eec-5384-420e-b05b-2eae2eae779d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:53.023635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af9eec-5384-420e-b05b-2eae2eae779d service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:53.023998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af9eec-5384-420e-b05b-2eae2eae779d service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:53.024364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af9eec-5384-420e-b05b-2eae2eae779d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:53.035855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af9eec-5384-420e-b05b-2eae2eae779d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:53.037349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af9eec-5384-420e-b05b-2eae2eae779d service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:53.047349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 728/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:55.221137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429b7fd-c8ce-4384-ae18-7433de373fa7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:55.222240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429b7fd-c8ce-4384-ae18-7433de373fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:55.222240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429b7fd-c8ce-4384-ae18-7433de373fa7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:55.224404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:09:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:55.230251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48256cd-373a-445c-9720-ded0d4e48fc5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:55.230784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48256cd-373a-445c-9720-ded0d4e48fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:55.230891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48256cd-373a-445c-9720-ded0d4e48fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:55.251092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d48256cd-373a-445c-9720-ded0d4e48fc5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:55.252307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d48256cd-373a-445c-9720-ded0d4e48fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:55.260889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:09:55.302094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429b7fd-c8ce-4384-ae18-7433de373fa7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:55.319995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429b7fd-c8ce-4384-ae18-7433de373fa7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:55.325658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fdfd-aee6-4d7e-b509-9c33f6b04c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:55.326500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fdfd-aee6-4d7e-b509-9c33f6b04c19 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:55.326917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fdfd-aee6-4d7e-b509-9c33f6b04c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:55.327260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fdfd-aee6-4d7e-b509-9c33f6b04c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:55.338397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fdfd-aee6-4d7e-b509-9c33f6b04c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:55.339962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fdfd-aee6-4d7e-b509-9c33f6b04c19 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:55.350034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 717/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:09:57.773747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d745f1-726c-4728-90a6-312009e241de None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:57.774295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d745f1-726c-4728-90a6-312009e241de None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:57.774295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d745f1-726c-4728-90a6-312009e241de None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:57.776536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:09:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:09:57.784869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941ec53-e2d8-45c8-9803-030d729f01f4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:57.785381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941ec53-e2d8-45c8-9803-030d729f01f4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:57.785381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941ec53-e2d8-45c8-9803-030d729f01f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:57.805047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c941ec53-e2d8-45c8-9803-030d729f01f4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:09:57.806253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c941ec53-e2d8-45c8-9803-030d729f01f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:09:57.815639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:09:57.854024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d745f1-726c-4728-90a6-312009e241de None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:09:57.865565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d745f1-726c-4728-90a6-312009e241de None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:09:57.869782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f98211-da6f-4856-907b-16d2540e2606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:09:57.870423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f98211-da6f-4856-907b-16d2540e2606 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:09:57.870725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f98211-da6f-4856-907b-16d2540e2606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:09:57.871022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f98211-da6f-4856-907b-16d2540e2606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:09:57.884664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f98211-da6f-4856-907b-16d2540e2606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:09:57.886934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f98211-da6f-4856-907b-16d2540e2606 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:09:57.899328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 731/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:00.078081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38f6fa-2168-4be9-8824-08b5ba85bf70 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:00.078655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38f6fa-2168-4be9-8824-08b5ba85bf70 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:00.078655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38f6fa-2168-4be9-8824-08b5ba85bf70 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:00.080014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:00.087122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa00a9-4df6-4e58-b512-c118984b554c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:00.087839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa00a9-4df6-4e58-b512-c118984b554c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:00.088020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa00a9-4df6-4e58-b512-c118984b554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:00.113732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-63fa00a9-4df6-4e58-b512-c118984b554c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:00.114946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-63fa00a9-4df6-4e58-b512-c118984b554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:00.123332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:00.162386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38f6fa-2168-4be9-8824-08b5ba85bf70 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:00.174340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38f6fa-2168-4be9-8824-08b5ba85bf70 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:00.179308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173d809-a098-4fc9-aa6e-d5046838c17f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:00.179866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d809-a098-4fc9-aa6e-d5046838c17f service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:00.180140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d809-a098-4fc9-aa6e-d5046838c17f service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:00.180411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d809-a098-4fc9-aa6e-d5046838c17f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:00.189973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173d809-a098-4fc9-aa6e-d5046838c17f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:00.191970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173d809-a098-4fc9-aa6e-d5046838c17f service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:00.201467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 720/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:02.318421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38ca7b-d6bf-4572-870d-42d4d4207c24 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:02.319031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38ca7b-d6bf-4572-870d-42d4d4207c24 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:02.319031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38ca7b-d6bf-4572-870d-42d4d4207c24 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:02.320376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:02.325837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06bdf8-690b-4f4b-9d1f-68f8c918b9fe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:02.326271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06bdf8-690b-4f4b-9d1f-68f8c918b9fe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:02.326367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06bdf8-690b-4f4b-9d1f-68f8c918b9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:02.345303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2e06bdf8-690b-4f4b-9d1f-68f8c918b9fe None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:02.346913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2e06bdf8-690b-4f4b-9d1f-68f8c918b9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:02.356250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:02.397363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38ca7b-d6bf-4572-870d-42d4d4207c24 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:02.409964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38ca7b-d6bf-4572-870d-42d4d4207c24 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:02.414043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec12ef-8138-4fa3-8967-5f6475b56cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:02.414747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec12ef-8138-4fa3-8967-5f6475b56cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:02.415038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec12ef-8138-4fa3-8967-5f6475b56cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:02.415317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec12ef-8138-4fa3-8967-5f6475b56cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:02.425120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec12ef-8138-4fa3-8967-5f6475b56cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:02.426605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec12ef-8138-4fa3-8967-5f6475b56cf0 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:02.436369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 734/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:10:04.542825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4106837-e4a5-49f8-b074-f0e4470b16ec None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:04.543410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4106837-e4a5-49f8-b074-f0e4470b16ec None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:04.543410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4106837-e4a5-49f8-b074-f0e4470b16ec None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:04.544754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:04.551397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450266f5-ec2b-4324-9018-8694e7f99580 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:04.552138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450266f5-ec2b-4324-9018-8694e7f99580 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:04.552290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450266f5-ec2b-4324-9018-8694e7f99580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:04.582306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-450266f5-ec2b-4324-9018-8694e7f99580 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:04.583549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-450266f5-ec2b-4324-9018-8694e7f99580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:04.592496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:04.638361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4106837-e4a5-49f8-b074-f0e4470b16ec None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:04.651834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4106837-e4a5-49f8-b074-f0e4470b16ec None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:04.657348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ba5a5-a5a4-4350-8321-ad6151af63df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:04.658272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ba5a5-a5a4-4350-8321-ad6151af63df service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:04.659294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ba5a5-a5a4-4350-8321-ad6151af63df service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:04.661494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ba5a5-a5a4-4350-8321-ad6151af63df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:04.677175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ba5a5-a5a4-4350-8321-ad6151af63df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:04.678642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ba5a5-a5a4-4350-8321-ad6151af63df service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:04.688796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 723/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:06.785786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3013ff-b82b-4a01-a87c-bda18c8cf587 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:06.786370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3013ff-b82b-4a01-a87c-bda18c8cf587 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:06.786370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3013ff-b82b-4a01-a87c-bda18c8cf587 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:06.787765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:06.794185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33213c02-cc9f-48a6-96dc-a1be1dda8020 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:06.794987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33213c02-cc9f-48a6-96dc-a1be1dda8020 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:06.795171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33213c02-cc9f-48a6-96dc-a1be1dda8020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:06.815612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-33213c02-cc9f-48a6-96dc-a1be1dda8020 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:06.816869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-33213c02-cc9f-48a6-96dc-a1be1dda8020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:06.825883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:06.866028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3013ff-b82b-4a01-a87c-bda18c8cf587 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:06.883163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3013ff-b82b-4a01-a87c-bda18c8cf587 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:06.888366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d3b1d-575a-4fc3-bf5c-5dd5cfe7c568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:06.889247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d3b1d-575a-4fc3-bf5c-5dd5cfe7c568 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:06.889570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d3b1d-575a-4fc3-bf5c-5dd5cfe7c568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:06.889845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d3b1d-575a-4fc3-bf5c-5dd5cfe7c568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:06.899887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d3b1d-575a-4fc3-bf5c-5dd5cfe7c568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:06.901328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d3b1d-575a-4fc3-bf5c-5dd5cfe7c568 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:06.912074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 737/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:08.977147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7051-b38d-4638-a686-789414aba796 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:08.977965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7051-b38d-4638-a686-789414aba796 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:08.977965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7051-b38d-4638-a686-789414aba796 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:08.980184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:08.986860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68160e4-32d0-4e3a-a094-e7b1c7df0eee None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:08.987299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68160e4-32d0-4e3a-a094-e7b1c7df0eee None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:08.987428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68160e4-32d0-4e3a-a094-e7b1c7df0eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:08.994078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=79 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:09.007561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a68160e4-32d0-4e3a-a094-e7b1c7df0eee None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:09.008788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a68160e4-32d0-4e3a-a094-e7b1c7df0eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:09.018468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:09.060162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c7051-b38d-4638-a686-789414aba796 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:09.070493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=73 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:09.081481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c7051-b38d-4638-a686-789414aba796 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:09.085721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b91c4-e1ca-49f1-9a12-2b1f03a179b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:09.086374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b91c4-e1ca-49f1-9a12-2b1f03a179b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:09.086656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b91c4-e1ca-49f1-9a12-2b1f03a179b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:09.086960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b91c4-e1ca-49f1-9a12-2b1f03a179b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:09.097672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b91c4-e1ca-49f1-9a12-2b1f03a179b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:09.099165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b91c4-e1ca-49f1-9a12-2b1f03a179b3 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:09.109478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 726/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:11.207921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230aa0-0623-40a7-a9e3-e35d6ac9d40d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:11.208833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230aa0-0623-40a7-a9e3-e35d6ac9d40d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:11.208833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230aa0-0623-40a7-a9e3-e35d6ac9d40d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:11.211108 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:11.220225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c7d0-b9bc-4599-a7e9-f43a9f939aed None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:11.221071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c7d0-b9bc-4599-a7e9-f43a9f939aed None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:11.221304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c7d0-b9bc-4599-a7e9-f43a9f939aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:11.246161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4783c7d0-b9bc-4599-a7e9-f43a9f939aed None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:11.247407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4783c7d0-b9bc-4599-a7e9-f43a9f939aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:11.256606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:11.296278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230aa0-0623-40a7-a9e3-e35d6ac9d40d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:11.310042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230aa0-0623-40a7-a9e3-e35d6ac9d40d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:11.314095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510a96a-ae57-4f34-a858-1917bb1f252e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:11.314860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a96a-ae57-4f34-a858-1917bb1f252e service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:11.315140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a96a-ae57-4f34-a858-1917bb1f252e service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:11.315415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510a96a-ae57-4f34-a858-1917bb1f252e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:11.325102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510a96a-ae57-4f34-a858-1917bb1f252e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:11.326587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510a96a-ae57-4f34-a858-1917bb1f252e service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:11.336848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 740/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:13.385237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d145d3-92ba-41ee-9555-f2738fac7f27 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:13.386185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d145d3-92ba-41ee-9555-f2738fac7f27 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:13.386185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d145d3-92ba-41ee-9555-f2738fac7f27 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:13.388329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:10:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:13.395742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1325c44-a5ea-4112-84f1-99294d317a0d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:13.396203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1325c44-a5ea-4112-84f1-99294d317a0d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:13.396309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1325c44-a5ea-4112-84f1-99294d317a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:13.415612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d1325c44-a5ea-4112-84f1-99294d317a0d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:13.416828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d1325c44-a5ea-4112-84f1-99294d317a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:13.426190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:13.466667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d145d3-92ba-41ee-9555-f2738fac7f27 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:13.485875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d145d3-92ba-41ee-9555-f2738fac7f27 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:13.490798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61d312-af23-43a5-ab01-f593307279c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:13.491533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61d312-af23-43a5-ab01-f593307279c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:13.491839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61d312-af23-43a5-ab01-f593307279c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:13.492182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61d312-af23-43a5-ab01-f593307279c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:13.502623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61d312-af23-43a5-ab01-f593307279c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:13.504143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61d312-af23-43a5-ab01-f593307279c3 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:13.514182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 729/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:23.409382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=11 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:23.509634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=14 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:33.409201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f1da-0a4e-40e4-af65-f11550c88cb8 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:33.409830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f1da-0a4e-40e4-af65-f11550c88cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:33.409830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661f1da-0a4e-40e4-af65-f11550c88cb8 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:33.411215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 742/1471] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:33.418107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdd9c-156f-47ab-bc52-e8360d4234a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:33.418553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdd9c-156f-47ab-bc52-e8360d4234a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:33.418666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdd9c-156f-47ab-bc52-e8360d4234a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:33.444848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b18bdd9c-156f-47ab-bc52-e8360d4234a8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:33.446059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b18bdd9c-156f-47ab-bc52-e8360d4234a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:33.455673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 730/1472] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:33.510519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673a7df-90ce-4382-ac52-7955ff22b25e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:33.510856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673a7df-90ce-4382-ac52-7955ff22b25e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:33.511138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673a7df-90ce-4382-ac52-7955ff22b25e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:33.512542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 743/1473] 10.222.0.64 () {54 vars in 962 bytes} [Fri Jun 24 10:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:33.517782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f00e8-0fa8-4ef6-9040-c4695d5ea15b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:33.518074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f00e8-0fa8-4ef6-9040-c4695d5ea15b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:33.518354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f00e8-0fa8-4ef6-9040-c4695d5ea15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:33.538251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-244f00e8-0fa8-4ef6-9040-c4695d5ea15b None None] MFA Rules not processed for user `bb0cbf8b93e741c2b500cbf1cb1ad5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:33.539494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-244f00e8-0fa8-4ef6-9040-c4695d5ea15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:33.547256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 731/1474] 10.222.0.64 () {58 vars in 1059 bytes} [Fri Jun 24 10:10:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:10:33.558474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673a7df-90ce-4382-ac52-7955ff22b25e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:33.576456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673a7df-90ce-4382-ac52-7955ff22b25e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:33.582322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e3d44-3171-48cb-9a22-7d981eb03133 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:33.583002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3d44-3171-48cb-9a22-7d981eb03133 service glance] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:33.583284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3d44-3171-48cb-9a22-7d981eb03133 service glance] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:33.583562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3d44-3171-48cb-9a22-7d981eb03133 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:33.593297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e3d44-3171-48cb-9a22-7d981eb03133 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:33.594752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e3d44-3171-48cb-9a22-7d981eb03133 service glance] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:33.604975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 744/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:35.754353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a1dbd-0cb1-4e2b-9f80-df0c1dbde538 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:35.755223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a1dbd-0cb1-4e2b-9f80-df0c1dbde538 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:35.755223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a1dbd-0cb1-4e2b-9f80-df0c1dbde538 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:35.757314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 732/1476] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:35.762749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badee07-cfb3-443f-8133-0b5e17495e33 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:35.763216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badee07-cfb3-443f-8133-0b5e17495e33 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:35.763309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badee07-cfb3-443f-8133-0b5e17495e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:35.783501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2badee07-cfb3-443f-8133-0b5e17495e33 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:35.784680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2badee07-cfb3-443f-8133-0b5e17495e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:35.793031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 745/1477] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:35.817237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a1dbd-0cb1-4e2b-9f80-df0c1dbde538 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:35.835258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a1dbd-0cb1-4e2b-9f80-df0c1dbde538 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:35.840603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:35.841253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:35.841517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:35.841781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:35.842512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:35.844415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:10:35.844415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:10:35.844415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:10:35.845790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020cc4b-dda7-455f-8872-fa6ac5ac2627 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:35.866134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 733/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Jun 24 10:10:35 2022] GET /identity/v3/projects => generated 2553 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:10:37.923892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864aa2-6dcc-47bf-9b01-eaf06862b9bf None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:37.924438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864aa2-6dcc-47bf-9b01-eaf06862b9bf None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:37.924544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864aa2-6dcc-47bf-9b01-eaf06862b9bf None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:37.925891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 746/1479] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:37.932341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19404ec-1c56-49e3-ae3a-0b4de50be64e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:37.933143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19404ec-1c56-49e3-ae3a-0b4de50be64e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:37.933313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19404ec-1c56-49e3-ae3a-0b4de50be64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:37.955313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f19404ec-1c56-49e3-ae3a-0b4de50be64e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:37.956529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f19404ec-1c56-49e3-ae3a-0b4de50be64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:37.965441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 734/1480] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:38.011698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864aa2-6dcc-47bf-9b01-eaf06862b9bf None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:38.028653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864aa2-6dcc-47bf-9b01-eaf06862b9bf None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:38.035957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b7437-46c7-4358-8a4f-1d0d659a3967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:38.036661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7437-46c7-4358-8a4f-1d0d659a3967 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:38.036944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7437-46c7-4358-8a4f-1d0d659a3967 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:38.037218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7437-46c7-4358-8a4f-1d0d659a3967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:38.046290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b7437-46c7-4358-8a4f-1d0d659a3967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:38.048013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b7437-46c7-4358-8a4f-1d0d659a3967 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:38.058249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 747/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:40.321519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f915bd-0848-49dc-b1b3-2d01117eb648 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:40.322375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f915bd-0848-49dc-b1b3-2d01117eb648 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:40.322375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f915bd-0848-49dc-b1b3-2d01117eb648 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:40.324378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 735/1482] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:40.330528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f681-1f58-4117-bf00-9097e377157c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:40.331100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f681-1f58-4117-bf00-9097e377157c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:40.331100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f681-1f58-4117-bf00-9097e377157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:40.350813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4d62f681-1f58-4117-bf00-9097e377157c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:40.352013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4d62f681-1f58-4117-bf00-9097e377157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:40.360713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 748/1483] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:40.397845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f915bd-0848-49dc-b1b3-2d01117eb648 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:40.414565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f915bd-0848-49dc-b1b3-2d01117eb648 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:40.418418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ae930-0f7d-40c2-8f62-91eddadd852d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:40.419075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ae930-0f7d-40c2-8f62-91eddadd852d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:40.419353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ae930-0f7d-40c2-8f62-91eddadd852d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:40.419619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ae930-0f7d-40c2-8f62-91eddadd852d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:40.430223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ae930-0f7d-40c2-8f62-91eddadd852d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:40.431717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ae930-0f7d-40c2-8f62-91eddadd852d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:40.441770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 736/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:42.509341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59920d90-f163-4466-b5bd-65b06332a2e1 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:42.509949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59920d90-f163-4466-b5bd-65b06332a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:42.509949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59920d90-f163-4466-b5bd-65b06332a2e1 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:42.512030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 749/1485] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:42.521377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49aa7-7f3a-4419-9f5a-8f4d5654e859 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:42.521856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49aa7-7f3a-4419-9f5a-8f4d5654e859 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:42.521948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49aa7-7f3a-4419-9f5a-8f4d5654e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:42.544717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-49a49aa7-7f3a-4419-9f5a-8f4d5654e859 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:42.545942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49a49aa7-7f3a-4419-9f5a-8f4d5654e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:42.554542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 737/1486] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:42.597015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59920d90-f163-4466-b5bd-65b06332a2e1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:42.614402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59920d90-f163-4466-b5bd-65b06332a2e1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:42.619044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5d5c4-3fab-40f6-9818-3706749196a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:42.619688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5d5c4-3fab-40f6-9818-3706749196a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:42.619967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5d5c4-3fab-40f6-9818-3706749196a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:42.620236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5d5c4-3fab-40f6-9818-3706749196a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:42.629644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5d5c4-3fab-40f6-9818-3706749196a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:42.631121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5d5c4-3fab-40f6-9818-3706749196a8 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:42.640584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 750/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:44.784813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a8d3-6ad2-44db-b697-ef62d0c29c1b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:44.785390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a8d3-6ad2-44db-b697-ef62d0c29c1b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:44.785390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a8d3-6ad2-44db-b697-ef62d0c29c1b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:44.786789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 738/1488] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:44.792427 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff537-31b3-4913-be5f-b22999ae2098 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:44.792880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff537-31b3-4913-be5f-b22999ae2098 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:44.792987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff537-31b3-4913-be5f-b22999ae2098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:44.812587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a21ff537-31b3-4913-be5f-b22999ae2098 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:44.814245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a21ff537-31b3-4913-be5f-b22999ae2098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:44.822971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 751/1489] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:10:44.864423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29a8d3-6ad2-44db-b697-ef62d0c29c1b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:44.881644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29a8d3-6ad2-44db-b697-ef62d0c29c1b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:44.888566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270a6dd-ffc3-4ef2-8e14-055dc77603e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:44.889475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270a6dd-ffc3-4ef2-8e14-055dc77603e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:44.889781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270a6dd-ffc3-4ef2-8e14-055dc77603e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:44.890083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270a6dd-ffc3-4ef2-8e14-055dc77603e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:44.900530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270a6dd-ffc3-4ef2-8e14-055dc77603e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:44.901990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270a6dd-ffc3-4ef2-8e14-055dc77603e1 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:44.911663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 739/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:47.207456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0a45-e830-4ff9-b5e4-d226d0618af1 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:47.208082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0a45-e830-4ff9-b5e4-d226d0618af1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:47.208082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0a45-e830-4ff9-b5e4-d226d0618af1 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:47.209386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 752/1491] 10.222.0.64 () {54 vars in 926 bytes} [Fri Jun 24 10:10:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:10:47.215335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558ec30-f55a-492d-bca1-ab2dac575890 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:47.215807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558ec30-f55a-492d-bca1-ab2dac575890 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:47.215917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558ec30-f55a-492d-bca1-ab2dac575890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:47.236927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6558ec30-f55a-492d-bca1-ab2dac575890 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:47.238358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6558ec30-f55a-492d-bca1-ab2dac575890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:47.248298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 740/1492] 10.222.0.64 () {58 vars in 1023 bytes} [Fri Jun 24 10:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:47.294436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0a45-e830-4ff9-b5e4-d226d0618af1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:47.310291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0a45-e830-4ff9-b5e4-d226d0618af1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:47.322437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91178d02-52dc-43e3-84cc-0388fd3c8f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:47.323132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91178d02-52dc-43e3-84cc-0388fd3c8f16 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:47.323423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91178d02-52dc-43e3-84cc-0388fd3c8f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:47.323703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91178d02-52dc-43e3-84cc-0388fd3c8f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:47.335996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91178d02-52dc-43e3-84cc-0388fd3c8f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:47.337452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91178d02-52dc-43e3-84cc-0388fd3c8f16 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:47.348734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 753/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:57.230593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=33 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:57.343144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=33 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:12:21.253646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.254598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.254598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.284024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:12:21.285587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:21.299808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 741/1494] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:12:21.309270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.309824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.309824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.330420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:12:21.331816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:21.340640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 754/1495] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:12:21.345272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.356516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25035d87-7ef1-460f-bb8c-480168a76e1d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.363073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.364026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.364423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.364811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.365974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:12:21.366283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.367892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:21.367892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:12:21.367892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:12:21.369181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.388002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 742/1496] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:12:21.396485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.406335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6d600-8bb8-4a47-8e77-a159e51e6e37 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.412548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.413208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.413496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.413747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.414456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.416318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:21.416318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:12:21.416318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:12:21.417543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.458821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 755/1497] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:12:21 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:12:21.463370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.474106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285823-5ad8-464d-87a8-aa47b5a5b5f5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.534656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.535313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.535612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.535884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.536570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.537600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:21.537600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:12:21.537600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:12:21.538819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.566684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 743/1498] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:12:21 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:12:21.572626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.583919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6a59-59be-411b-aabd-4cdbbf4b1b4f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.589091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.589755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.590036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.590313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.591224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.592239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:21.592239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:12:21.592239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:12:21.593143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.606728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 756/1499] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:12:21.610816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.621676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41108a0c-645a-4527-aa2f-20e1ee1ae9e9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.627229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.628025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.628301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.628548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.629311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.630339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:21.630339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:12:21.630339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:12:21.631263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56d747-180a-4c55-92e0-61be83edf11e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.645260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 744/1500] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:12:21.649664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.660920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10e99-abf2-4e73-8b1f-43ae5926a632 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.667396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.668287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.668561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.668836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] PATH_INFO: `/v3/projects/53191d9c0ec149a4b32b97875b738ed7/users/d11dd60b38714991b8b9972d6b44ded8/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.686833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=53191d9c0ec149a4b32b97875b738ed7, user_id=d11dd60b38714991b8b9972d6b44ded8, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.687884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:21.687884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:12:21.687884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:12:21.689878 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd3bec-2507-403a-8775-9891c3a337fa admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.707934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 757/1501] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:12:21 2022] PUT /identity/v3/projects/53191d9c0ec149a4b32b97875b738ed7/users/d11dd60b38714991b8b9972d6b44ded8/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:12:21.720136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe40c6-ea99-4e08-99c4-7dddaf6d3cb7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.720427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe40c6-ea99-4e08-99c4-7dddaf6d3cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.720706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe40c6-ea99-4e08-99c4-7dddaf6d3cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.752639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-90fe40c6-ea99-4e08-99c4-7dddaf6d3cb7 None None] MFA Rules not processed for user `d11dd60b38714991b8b9972d6b44ded8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:12:21.754043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-90fe40c6-ea99-4e08-99c4-7dddaf6d3cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:21.827432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 745/1502] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:12:21.842169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e932-f4a1-4e6a-b6cb-fa57e754bf3d None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.842461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e932-f4a1-4e6a-b6cb-fa57e754bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.842761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e932-f4a1-4e6a-b6cb-fa57e754bf3d None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.844147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 758/1503] 10.222.0.64 () {54 vars in 963 bytes} [Fri Jun 24 10:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:12:21.849246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0413e-d52b-481e-bd05-c69fc94c0a57 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.849527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0413e-d52b-481e-bd05-c69fc94c0a57 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.849803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0413e-d52b-481e-bd05-c69fc94c0a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.869593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-38e0413e-d52b-481e-bd05-c69fc94c0a57 None None] MFA Rules not processed for user `dfd4560d3f144b339adb62b9ddf118ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:12:21.870855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-38e0413e-d52b-481e-bd05-c69fc94c0a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:21.919830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:12:21.929359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 746/1504] 10.222.0.64 () {58 vars in 1060 bytes} [Fri Jun 24 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:12:21.937279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e932-f4a1-4e6a-b6cb-fa57e754bf3d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:21.948445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e932-f4a1-4e6a-b6cb-fa57e754bf3d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:21.952400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:21.953096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:21.953419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:21.953695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:21.962645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:21.964124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:21.973460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 759/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:12:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:12:22.019689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30673-ee35-449e-842e-54e55a65aecf None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:22.019980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30673-ee35-449e-842e-54e55a65aecf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:22.020256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30673-ee35-449e-842e-54e55a65aecf None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:22.021593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 747/1506] 10.222.0.64 () {52 vars in 829 bytes} [Fri Jun 24 10:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:12:22.025459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:22.093950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f439b-4c55-44c4-a28f-84b5d0d270ef service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:22.099865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:22.100747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:22.101027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:22.101302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] PATH_INFO: `/v3/users/d11dd60b38714991b8b9972d6b44ded8` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:22.102622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11dd60b38714991b8b9972d6b44ded8)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:22.103689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:22.103689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:12:22.103689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:12:22.104918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:22.173323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11dd60b38714991b8b9972d6b44ded8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.191645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11dd60b38714991b8b9972d6b44ded8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.203117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11dd60b38714991b8b9972d6b44ded8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.212120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:12:22.217234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11dd60b38714991b8b9972d6b44ded8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.217970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11dd60b38714991b8b9972d6b44ded8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.225752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 760/1507] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:12:22 2022] DELETE /identity/v3/users/d11dd60b38714991b8b9972d6b44ded8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:12:22.230310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:22.230604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:22.230914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:22.250440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:12:22.251724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:22.318154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 748/1508] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:12:22.328659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:22.330456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:22.402179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04195cf7-5564-44e7-8d62-785013f38476 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:22.409225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb386b-fbf0-4b3e-899a-35071ad0d0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:22.409876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb386b-fbf0-4b3e-899a-35071ad0d0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:22.410159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb386b-fbf0-4b3e-899a-35071ad0d0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:22.410431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb386b-fbf0-4b3e-899a-35071ad0d0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:22.421184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb386b-fbf0-4b3e-899a-35071ad0d0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:22.422646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb386b-fbf0-4b3e-899a-35071ad0d0f1 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:22.433130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 761/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:12:22.804678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:12:22.806158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:12:22.818978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc43850-e5fb-4bc4-85be-22a9d9ba7ecf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:12:22.823478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:12:22.824338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:12:22.824637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:12:22.824879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] PATH_INFO: `/v3/projects/53191d9c0ec149a4b32b97875b738ed7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:12:22.826083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53191d9c0ec149a4b32b97875b738ed7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:12:22.827096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:12:22.827096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:12:22.827096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:12:22.828320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:12:22.894333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53191d9c0ec149a4b32b97875b738ed7', 'request_id': 'req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.901418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53191d9c0ec149a4b32b97875b738ed7', 'request_id': 'req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.923759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53191d9c0ec149a4b32b97875b738ed7', 'request_id': 'req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.941870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] The token cache is being invalidate because project 53191d9c0ec149a4b32b97875b738ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:12:22.942158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.942934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e0adf50-cd02-4703-a6b7-74fc5d1c6e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:12:22.945177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 749/1510] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:12:22 2022] DELETE /identity/v3/projects/53191d9c0ec149a4b32b97875b738ed7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:12:32.428671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=29 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:12:32.938065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:14:04.359832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c32c3-0047-4bcd-b586-3f0047507de4 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:04.360754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c32c3-0047-4bcd-b586-3f0047507de4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:04.360754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c32c3-0047-4bcd-b586-3f0047507de4 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:04.362962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:04.371337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e9ad2-2366-42e1-8051-3b5d50abccc8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:04.371782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e9ad2-2366-42e1-8051-3b5d50abccc8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:04.371910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e9ad2-2366-42e1-8051-3b5d50abccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:04.393354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-281e9ad2-2366-42e1-8051-3b5d50abccc8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:04.394594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-281e9ad2-2366-42e1-8051-3b5d50abccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:04.460444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:04.618308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c32c3-0047-4bcd-b586-3f0047507de4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:04.625873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2e7c32c3-0047-4bcd-b586-3f0047507de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:04.712778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c32c3-0047-4bcd-b586-3f0047507de4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:04.718646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c58740-0c13-45a0-aa6e-2b05789c2698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:04.719686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c58740-0c13-45a0-aa6e-2b05789c2698 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:04.719968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c58740-0c13-45a0-aa6e-2b05789c2698 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:04.720244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c58740-0c13-45a0-aa6e-2b05789c2698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:04.728579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c58740-0c13-45a0-aa6e-2b05789c2698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:04.730193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c58740-0c13-45a0-aa6e-2b05789c2698 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:04.739842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 763/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:05.588465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3e07c-173f-4b70-94c7-9cc410916809 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:05.589324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3e07c-173f-4b70-94c7-9cc410916809 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:05.589324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3e07c-173f-4b70-94c7-9cc410916809 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:05.591565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:14:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:05.597090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2e07a-9736-4760-93d0-1b67b76a8abd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:05.597552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2e07a-9736-4760-93d0-1b67b76a8abd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:05.597655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2e07a-9736-4760-93d0-1b67b76a8abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:05.618643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f5a2e07a-9736-4760-93d0-1b67b76a8abd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:05.619883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f5a2e07a-9736-4760-93d0-1b67b76a8abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:05.628494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:05.783800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3e07c-173f-4b70-94c7-9cc410916809 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:05.799605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3e07c-173f-4b70-94c7-9cc410916809 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:05.806053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3e03a-4275-400d-97ae-6a41ea21e55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:05.806689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3e03a-4275-400d-97ae-6a41ea21e55d service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:05.806994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3e03a-4275-400d-97ae-6a41ea21e55d service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:05.807261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3e03a-4275-400d-97ae-6a41ea21e55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:05.817050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3e03a-4275-400d-97ae-6a41ea21e55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:05.818749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3e03a-4275-400d-97ae-6a41ea21e55d service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:05.828968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 752/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:06.258122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1255fb-67ce-4e9c-8460-fafd02418b3c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:06.258830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1255fb-67ce-4e9c-8460-fafd02418b3c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:06.259125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1255fb-67ce-4e9c-8460-fafd02418b3c None None] PATH_INFO: `/v3` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:06.260468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 765/1517] 10.222.0.27 () {54 vars in 925 bytes} [Fri Jun 24 10:14:06 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:14:06.267161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41497bc-4f91-41a6-a926-ada2f6cd293a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:06.267453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41497bc-4f91-41a6-a926-ada2f6cd293a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:06.267717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41497bc-4f91-41a6-a926-ada2f6cd293a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:06.288544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b41497bc-4f91-41a6-a926-ada2f6cd293a None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:06.289785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b41497bc-4f91-41a6-a926-ada2f6cd293a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:06.363918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 753/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Jun 24 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:10.140747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e918e45-8ad4-45e9-a29a-c24f4212c946 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:10.141337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e918e45-8ad4-45e9-a29a-c24f4212c946 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:10.141337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e918e45-8ad4-45e9-a29a-c24f4212c946 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:10.143302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 766/1519] 10.222.0.64 () {54 vars in 960 bytes} [Fri Jun 24 10:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:10.148394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc82b52-12ee-4e61-8a90-4b689c8eeda4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:10.148852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc82b52-12ee-4e61-8a90-4b689c8eeda4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:10.148949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc82b52-12ee-4e61-8a90-4b689c8eeda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:10.167665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0dc82b52-12ee-4e61-8a90-4b689c8eeda4 None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:10.168860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0dc82b52-12ee-4e61-8a90-4b689c8eeda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:10.178394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 754/1520] 10.222.0.64 () {58 vars in 1057 bytes} [Fri Jun 24 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:10.186020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e918e45-8ad4-45e9-a29a-c24f4212c946 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:10.203046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e918e45-8ad4-45e9-a29a-c24f4212c946 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:10.208924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50f45-0334-4616-844b-b6a3700dfb7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:10.209554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50f45-0334-4616-844b-b6a3700dfb7c service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:10.209834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50f45-0334-4616-844b-b6a3700dfb7c service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:10.210106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50f45-0334-4616-844b-b6a3700dfb7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:10.220019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50f45-0334-4616-844b-b6a3700dfb7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:10.221612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50f45-0334-4616-844b-b6a3700dfb7c service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:10.231540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 767/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:17.064141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f39c9-a109-4533-93cb-116fdd01c40f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:17.064713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f39c9-a109-4533-93cb-116fdd01c40f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:17.064713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f39c9-a109-4533-93cb-116fdd01c40f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:17.066021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:17.072159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d29386-3273-4c3c-9421-4efc242a7a94 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:17.072914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d29386-3273-4c3c-9421-4efc242a7a94 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:17.073024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d29386-3273-4c3c-9421-4efc242a7a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:17.100922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-17d29386-3273-4c3c-9421-4efc242a7a94 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:17.102121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-17d29386-3273-4c3c-9421-4efc242a7a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:17.112064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:17.280368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f39c9-a109-4533-93cb-116fdd01c40f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:17.303534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f39c9-a109-4533-93cb-116fdd01c40f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:17.310052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1375c32-2cff-4ae6-bec2-82aed3810489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:17.310689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1375c32-2cff-4ae6-bec2-82aed3810489 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:17.310994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1375c32-2cff-4ae6-bec2-82aed3810489 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:17.311262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1375c32-2cff-4ae6-bec2-82aed3810489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:17.320949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1375c32-2cff-4ae6-bec2-82aed3810489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:17.322517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1375c32-2cff-4ae6-bec2-82aed3810489 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:17.332367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 756/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:18.158749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859c2b-dfd6-495a-b167-640e9f09634c None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:18.159358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859c2b-dfd6-495a-b167-640e9f09634c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:18.159358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859c2b-dfd6-495a-b167-640e9f09634c None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:18.160685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 10:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:18.166274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5c80-43ed-4e38-bf84-53fe8cea8063 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:18.166754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5c80-43ed-4e38-bf84-53fe8cea8063 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:18.166870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb5c80-43ed-4e38-bf84-53fe8cea8063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:18.188684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f3cb5c80-43ed-4e38-bf84-53fe8cea8063 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:18.189849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3cb5c80-43ed-4e38-bf84-53fe8cea8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:18.198627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:18.362361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9859c2b-dfd6-495a-b167-640e9f09634c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:18.384418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9859c2b-dfd6-495a-b167-640e9f09634c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:18.390679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049650b-5a14-4103-a310-cd2f540c1415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:18.391349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049650b-5a14-4103-a310-cd2f540c1415 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:18.391726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049650b-5a14-4103-a310-cd2f540c1415 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:18.391973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049650b-5a14-4103-a310-cd2f540c1415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:18.400262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049650b-5a14-4103-a310-cd2f540c1415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:18.401885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049650b-5a14-4103-a310-cd2f540c1415 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:18.412039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 770/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:22.268556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cb7bc-a322-46d1-be9d-d0daa3d0ed26 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:22.269164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cb7bc-a322-46d1-be9d-d0daa3d0ed26 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:22.269164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cb7bc-a322-46d1-be9d-d0daa3d0ed26 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:22.270505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:14:22.276421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe267-82ce-4134-97a3-f0fb16070261 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:22.276891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe267-82ce-4134-97a3-f0fb16070261 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:22.276999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fe267-82ce-4134-97a3-f0fb16070261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:22.298217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-df1fe267-82ce-4134-97a3-f0fb16070261 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:22.299531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-df1fe267-82ce-4134-97a3-f0fb16070261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:22.308555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:22.353048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cb7bc-a322-46d1-be9d-d0daa3d0ed26 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:22.354973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9d2cb7bc-a322-46d1-be9d-d0daa3d0ed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:22.428456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cb7bc-a322-46d1-be9d-d0daa3d0ed26 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:22.432419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cfcb8-56ab-4583-abe8-e2ad07eeed14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:22.433065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cfcb8-56ab-4583-abe8-e2ad07eeed14 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:22.433347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cfcb8-56ab-4583-abe8-e2ad07eeed14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:22.433621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cfcb8-56ab-4583-abe8-e2ad07eeed14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:22.442808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cfcb8-56ab-4583-abe8-e2ad07eeed14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:22.444254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cfcb8-56ab-4583-abe8-e2ad07eeed14 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:22.449142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:14:22.454612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 759/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:24.479369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d269b-4d50-474d-a084-95812d44ab5a None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:24.479969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d269b-4d50-474d-a084-95812d44ab5a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:24.479969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d269b-4d50-474d-a084-95812d44ab5a None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:24.481886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:24.489513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7965-d9df-4282-8e57-0fcb48f1b72a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:24.490021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7965-d9df-4282-8e57-0fcb48f1b72a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:24.490021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7965-d9df-4282-8e57-0fcb48f1b72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:24.511350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e30b7965-d9df-4282-8e57-0fcb48f1b72a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:24.512560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e30b7965-d9df-4282-8e57-0fcb48f1b72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:24.521098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:24.557435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d269b-4d50-474d-a084-95812d44ab5a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:24.559468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-705d269b-4d50-474d-a084-95812d44ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:24.573929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d269b-4d50-474d-a084-95812d44ab5a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:24.577891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f1889-5bb5-4359-bb56-571cc6ed8ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:24.578578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1889-5bb5-4359-bb56-571cc6ed8ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:24.578903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1889-5bb5-4359-bb56-571cc6ed8ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:24.579398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1889-5bb5-4359-bb56-571cc6ed8ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:24.589336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f1889-5bb5-4359-bb56-571cc6ed8ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:24.590772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f1889-5bb5-4359-bb56-571cc6ed8ba9 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:24.601097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 773/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:28.287848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f49c0f-072e-44b0-a11d-f3738afb3390 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:28.288441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f49c0f-072e-44b0-a11d-f3738afb3390 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:28.288674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f49c0f-072e-44b0-a11d-f3738afb3390 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:28.290365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:28.296920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72e78-859b-4161-a86c-dea8d7b24bfc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:28.297470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72e78-859b-4161-a86c-dea8d7b24bfc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:28.297860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72e78-859b-4161-a86c-dea8d7b24bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:28.319010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-99b72e78-859b-4161-a86c-dea8d7b24bfc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:28.320226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-99b72e78-859b-4161-a86c-dea8d7b24bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:28.329472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:28.635429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f49c0f-072e-44b0-a11d-f3738afb3390 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:28.638089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e4f49c0f-072e-44b0-a11d-f3738afb3390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:28.705006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f49c0f-072e-44b0-a11d-f3738afb3390 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:28.709205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd5c17-0457-4f9f-8314-411ba3f94923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:28.709940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5c17-0457-4f9f-8314-411ba3f94923 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:28.710129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5c17-0457-4f9f-8314-411ba3f94923 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:28.710406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5c17-0457-4f9f-8314-411ba3f94923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:28.721123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd5c17-0457-4f9f-8314-411ba3f94923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:28.722579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd5c17-0457-4f9f-8314-411ba3f94923 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:28.732437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 762/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:30.808149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f755c0-8443-4300-94ce-ae997e5b7530 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:30.808749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f755c0-8443-4300-94ce-ae997e5b7530 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:30.808749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f755c0-8443-4300-94ce-ae997e5b7530 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:30.810094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 10:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:14:30.816793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b60e9d-c667-4f00-b58a-05b516ff59d5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:30.817567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b60e9d-c667-4f00-b58a-05b516ff59d5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:30.817744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b60e9d-c667-4f00-b58a-05b516ff59d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:30.841886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-81b60e9d-c667-4f00-b58a-05b516ff59d5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:14:30.843166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-81b60e9d-c667-4f00-b58a-05b516ff59d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:30.852622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 10:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:14:31.159222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f755c0-8443-4300-94ce-ae997e5b7530 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:14:31.161211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-14f755c0-8443-4300-94ce-ae997e5b7530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:14:31.175523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f755c0-8443-4300-94ce-ae997e5b7530 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:14:31.179654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1bfbb-e4d9-4d28-abd4-8a37b7af3f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:14:31.180300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1bfbb-e4d9-4d28-abd4-8a37b7af3f51 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:14:31.180590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1bfbb-e4d9-4d28-abd4-8a37b7af3f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:14:31.180859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1bfbb-e4d9-4d28-abd4-8a37b7af3f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:14:31.190653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1bfbb-e4d9-4d28-abd4-8a37b7af3f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:14:31.192157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1bfbb-e4d9-4d28-abd4-8a37b7af3f51 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:14:31.201867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 776/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:14:40.836595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=32 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:14:41.197836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=70 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:15:58.854340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46506fe-93f2-440e-907f-69b5b152847b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:58.855477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46506fe-93f2-440e-907f-69b5b152847b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:58.856064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46506fe-93f2-440e-907f-69b5b152847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:58.879955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a46506fe-93f2-440e-907f-69b5b152847b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:58.881510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a46506fe-93f2-440e-907f-69b5b152847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:58.893576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 764/1540] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:15:58.896169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97baca8-6944-49fc-b1f3-95231d33121b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:58.896729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97baca8-6944-49fc-b1f3-95231d33121b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:58.896729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97baca8-6944-49fc-b1f3-95231d33121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:58.914212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:58.914804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:58.915298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:58.929965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f97baca8-6944-49fc-b1f3-95231d33121b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:58.931747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f97baca8-6944-49fc-b1f3-95231d33121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:58.942750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:58.944196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:58.951460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 777/1541] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:15:58.960393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 765/1542] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:15:58.961636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:58.962181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:58.962944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:58.969554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:58.985174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8e611-d7d2-4bd6-bcbf-c0a9536cb277 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:58.989286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:58.995286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:58.995649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:58.996514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:58.996949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:58.997373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:58.998395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:58.998879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.001121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.008579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 778/1543] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.013911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.024985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 766/1544] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:15:59.028767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235726b6-5daa-4850-ad6b-aa991c9ad7e3 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.035702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.043393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.044173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.044739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.045092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.045909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.046313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.047588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:15:59.047588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:15:59.047588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:15:59.048575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.052449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e134-ac7d-4784-b3f4-a6e85a70fc2b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.062983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 779/1545] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.070576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.073768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.074746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.075268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.075747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.076644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.077888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:15:59.077888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:15:59.077888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:15:59.079496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c36608-93b9-48ad-b8aa-ea0fd72bf630 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.083178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ee9ed-c635-4a33-aa31-ced6591806ef admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.098087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023f937-b8bd-4e3d-a4ee-4713455f2102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.098938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023f937-b8bd-4e3d-a4ee-4713455f2102 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.099227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023f937-b8bd-4e3d-a4ee-4713455f2102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.099509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023f937-b8bd-4e3d-a4ee-4713455f2102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.100228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023f937-b8bd-4e3d-a4ee-4713455f2102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.102189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023f937-b8bd-4e3d-a4ee-4713455f2102 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.151029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 767/1546] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:15:59.152482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed6da-129a-4d38-a7a9-d3d15f8f0674 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.152781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed6da-129a-4d38-a7a9-d3d15f8f0674 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.153060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed6da-129a-4d38-a7a9-d3d15f8f0674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.155112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 780/1547] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:15:59.157109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.157405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.157686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.184014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c61ed6da-129a-4d38-a7a9-d3d15f8f0674 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.187879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c61ed6da-129a-4d38-a7a9-d3d15f8f0674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.197592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.198830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.308174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 768/1548] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.309430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.309720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.309999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.330565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 781/1549] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.331663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.337177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.338464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.354946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebf606-7f0f-41d8-9659-4d923e130ff6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.358089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 769/1550] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.359153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.367274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.368470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.368983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.369486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.369800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b3a15-2f9c-45dc-bb7f-561082edc751 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.370891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.372847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:15:59.372847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:15:59.372847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:15:59.375185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a45f0e-fbfb-435d-a6c0-d81d077252a0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.375527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e431665-f698-4f3b-aa56-ff975f3d6165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.376212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431665-f698-4f3b-aa56-ff975f3d6165 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.376494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431665-f698-4f3b-aa56-ff975f3d6165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.376772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431665-f698-4f3b-aa56-ff975f3d6165 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.377474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e431665-f698-4f3b-aa56-ff975f3d6165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.379469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e431665-f698-4f3b-aa56-ff975f3d6165 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.406871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 770/1551] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:15:59.408138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 782/1552] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/users => generated 285 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:15:59.408261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c599-1768-4774-8322-cc7a04daaf05 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.408463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c599-1768-4774-8322-cc7a04daaf05 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.408745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c599-1768-4774-8322-cc7a04daaf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.409782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e586618-e862-4c26-b2aa-f06131f25626 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.410241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e586618-e862-4c26-b2aa-f06131f25626 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.410582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e586618-e862-4c26-b2aa-f06131f25626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.427344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6f28c599-1768-4774-8322-cc7a04daaf05 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.428375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9e586618-e862-4c26-b2aa-f06131f25626 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.428544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6f28c599-1768-4774-8322-cc7a04daaf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.429563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9e586618-e862-4c26-b2aa-f06131f25626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.436496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 771/1553] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.437843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bac6f-2f5c-4d73-9da0-5ffcce307f0b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.438023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 783/1554] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.439503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17cbe-73ec-4353-aefe-fe7675a88871 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.439806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17cbe-73ec-4353-aefe-fe7675a88871 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.440111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17cbe-73ec-4353-aefe-fe7675a88871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.441557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bac6f-2f5c-4d73-9da0-5ffcce307f0b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.441849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bac6f-2f5c-4d73-9da0-5ffcce307f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.459986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4f7bac6f-2f5c-4d73-9da0-5ffcce307f0b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.461211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4f7bac6f-2f5c-4d73-9da0-5ffcce307f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.463905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-93f17cbe-73ec-4353-aefe-fe7675a88871 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.465102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-93f17cbe-73ec-4353-aefe-fe7675a88871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.469934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 772/1555] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.471194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.471483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.471759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.474962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 784/1556] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.476295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.476552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.476841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.491456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.492674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.494978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.496146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.500739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 773/1557] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.501512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.505269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 785/1558] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.506779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.513302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb6059-75dd-4724-9a1c-af7e60d12eac None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.519008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.519163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858363f-ee7b-4b7a-8117-abba68be228b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.519693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.519979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.520258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.521003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.522644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.523695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099e3c1-a1cd-4864-9cb3-81b289488b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.524361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099e3c1-a1cd-4864-9cb3-81b289488b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.524690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099e3c1-a1cd-4864-9cb3-81b289488b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.525001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099e3c1-a1cd-4864-9cb3-81b289488b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.525792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099e3c1-a1cd-4864-9cb3-81b289488b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.527431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099e3c1-a1cd-4864-9cb3-81b289488b71 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.536543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 774/1559] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:15:59.536543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.540292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 786/1560] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:15:59.541640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.541912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.542213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.544846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb516aab-be90-45f5-b2d1-0382efbc5152 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.549732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.550384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.550669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.550975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.551701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.552005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.553642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.559857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.560990 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.566797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 775/1561] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.567664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.569440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 787/1562] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.570411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.578319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84009a-4974-459a-a53c-b8373cae6d61 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.580916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c58add-9353-436d-bee6-b37e65651085 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.582909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.583553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.583835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.584106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.584822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.585124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.585575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.586215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.586496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.586746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58271b83-7193-4d47-8e94-226d54bb680b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.586916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.588323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.588498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.590087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595021bd-fc52-4769-8cb2-af24ef63a7c6 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.599169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 776/1563] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.600400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.600688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.600965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.602081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 788/1564] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.610873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.611158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.611441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.619663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.620882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.628894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 777/1565] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.629677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.629965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:15:59.631183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:15:59.639864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c32f51-482c-468c-bad5-1cf217f38f74 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.640137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 789/1566] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:15:59.641062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.644359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.645012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.645292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.645565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.646290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.647951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.650894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1bda9-5b09-49e6-bf18-394c87b5792e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.655584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.656235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.656515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.656788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.657690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.661487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 778/1567] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:15:59.661487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.661724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.670825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 790/1568] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:15:59.671871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930224f4-ea22-480a-b417-fe8e3163e9de admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.672388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.676924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.677581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.677864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.678159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.678915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.679221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.681094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.682612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271407d-b01e-40df-89b2-191acac36aa9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.687729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.688520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.688907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.689169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.689975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.690678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:15:59.692537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.697968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 779/1569] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.699086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.715905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70c31e-b1bd-4fce-a54b-90b65a3e9bb2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.721545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.722302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.722584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.722898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.723587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.725496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.726263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 791/1570] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:15:59.727177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.732854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:15:59.755920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 780/1571] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:15:59.755920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.793746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16b8ad-0f97-4eaa-8184-0a8a4a7b0d59 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.817964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497af38f-a06c-4457-b9ca-0c1fd85c04fb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.823045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.823709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.823988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.824259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.824975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.825281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.826905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.838594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 781/1572] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.846833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.851564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.852242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.852531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.853328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.853562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.855500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.856824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78625-02cb-40d7-a096-1c2ed855881c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.861291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.861931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.862207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.862477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.863207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:15:59.863505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.865063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.877932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 782/1573] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:15:59.878762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.886066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 792/1574] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:15:59.887317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.889280 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2ad2b-1ae9-4d74-a42d-ea880fc90322 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.952001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ebd86-9fde-434b-82df-bfe5259f3a04 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:15:59.952620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.953163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.953443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.953719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] PATH_INFO: `/v3/projects/1f1538c924054dc694f1207737d55dda/users/54c73c484dcc4b5686a9226979ad8fc9/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.957448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:15:59.958417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:15:59.958816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:15:59.959207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] PATH_INFO: `/v3/projects/74111042535d44c8a52a4656eb37fa23/users/0a66b13572904a309d3180afc7278070/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:15:59.969200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1538c924054dc694f1207737d55dda, user_id=54c73c484dcc4b5686a9226979ad8fc9, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.970484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:15:59.970484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:15:59.970484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:15:59.972448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.975752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74111042535d44c8a52a4656eb37fa23, user_id=0a66b13572904a309d3180afc7278070, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:15:59.979255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:15:59.987226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 783/1575] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:15:59 2022] PUT /identity/v3/projects/1f1538c924054dc694f1207737d55dda/users/54c73c484dcc4b5686a9226979ad8fc9/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:15:59.988092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:15:59.996332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 793/1576] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:15:59 2022] PUT /identity/v3/projects/74111042535d44c8a52a4656eb37fa23/users/0a66b13572904a309d3180afc7278070/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:15:59.996332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.018135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:00.033694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:00.052688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70290b94-2dba-4b6c-9f84-09b089fc017e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.057404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.058063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.058466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.058770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.059465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.060675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22989b5-a63b-48df-acf0-68a75224d743 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.061407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.065301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.065976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.066258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.066561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.067307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.069294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.090525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 784/1577] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.094608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 794/1578] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:15:59 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.094791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.094909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.103907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23885a2b-aee9-4fcb-9bf4-eb4a0f4eb5a8 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.160385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.161087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.161372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.161651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.162355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.162784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2522ac7-69f0-439a-9969-f6c725fe60f1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.164415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.167446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.168151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.168400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.168672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.169364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.171358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.191813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 785/1579] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.192682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.193060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 795/1580] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.193931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.203491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5907d-3470-41d6-a819-d74f5f341c20 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.258454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50858-84eb-417f-b855-e0020cf0814d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.260373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b36e7-6b0c-4125-8285-526353992f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.261047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b36e7-6b0c-4125-8285-526353992f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.261328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b36e7-6b0c-4125-8285-526353992f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.261652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b36e7-6b0c-4125-8285-526353992f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.262302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b36e7-6b0c-4125-8285-526353992f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.263185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58acfda-fc8d-4dd7-a0e4-b587cf9fdea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.263861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58acfda-fc8d-4dd7-a0e4-b587cf9fdea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.264147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58acfda-fc8d-4dd7-a0e4-b587cf9fdea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.264263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b36e7-6b0c-4125-8285-526353992f41 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.264427 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58acfda-fc8d-4dd7-a0e4-b587cf9fdea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.265117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58acfda-fc8d-4dd7-a0e4-b587cf9fdea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.267173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58acfda-fc8d-4dd7-a0e4-b587cf9fdea8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.292345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 796/1581] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.292577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 786/1582] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.292669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.292669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.292669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.292896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.292896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.292896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.311044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:00.312455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:00.319519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] MFA Rules not processed for user `54c73c484dcc4b5686a9226979ad8fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:00.320708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:00.331127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:00.350906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:00.373692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 797/1583] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:00.374800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.385049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada45cba-1ccb-44ce-8d7e-a7c574efe70a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.388179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 787/1584] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:00.389014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.390198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.390901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.391188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.391460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.392166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.394095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.399223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76742f2-a88b-4796-bf60-da1e5285c148 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.404417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.405033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.405322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.405649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.406330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.407973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.417230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 798/1585] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.418075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.418647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 788/1586] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.422139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.428264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99577e-8bbc-439a-9759-aa004545e87b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.431753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51574538-176c-456d-bab8-e3aafd67fa5e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.433799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.434466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.434790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.435071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.436227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.436530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.437179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.437463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.437741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.438444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.438551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.440059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.451625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 789/1587] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.452755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.462227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 799/1588] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.462627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690db002-e1a6-4a36-bdf0-017611e36818 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.463172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.468229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.468888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.469167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.469438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.470160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.471801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.473407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd18852-6634-43c3-88e5-9ea956e18e37 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.477976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.478632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.478950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.479230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.479919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.481831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.482458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 790/1589] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.483351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.502110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 800/1590] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/users => generated 290 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.503084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.512847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd468d2-6873-4d24-b396-43d37e3f8a27 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.518020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.518668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.519142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.519419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.520125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.522070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.537006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f25d-bd8b-48d6-89cd-27fc29aef77b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.540991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.541692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.542000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.542320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.549538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 801/1591] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.550596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.551604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.553056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.563191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 791/1592] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:00.563983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.573417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e2cc9-8024-4ee7-97ea-148531b6dc14 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.624105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46280f0c-9cac-4acb-859d-191681090a0d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.628849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.629533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.629813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.630085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.631035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.633165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.636149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.636835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.637051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.637298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.638058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.639715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.645879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 802/1593] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.646873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.656534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 792/1594] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.656534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.662162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67f7bd-2b90-4bd5-89d1-2e88c42b3217 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.667533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5cad6-2494-42d5-8dd2-323f2c3130e7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.670305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.671479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.671967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.672244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.672673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.673110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.673341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.673624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.673899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.674637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.674836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.676484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.686986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 803/1595] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.688012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.688838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 793/1596] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.690036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.702812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b252b77-f3bd-4f5a-8f1e-c2417f388867 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.706242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56b42-5627-44e3-8260-6e41585aedc3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.709728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.710872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.711675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.712172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.712963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.713180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.713805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.714103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.714288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.715251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.715976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.718112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.732201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 804/1597] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.732201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.739036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 794/1598] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.739036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.740368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8334b5-cb04-45fd-bead-34e005846b9e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.746172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.746859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.747135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.747465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.748170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.750264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8ffd-b78a-4ab3-a5cc-ec64764d4e69 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.750424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.756406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.756825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.757167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.757506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.758299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.760843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.771989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 805/1599] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:00 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:00.773004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 795/1600] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.773365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.774022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.781607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:00.784806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df615a-f83a-4911-b9d0-8c518c45a280 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.787489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62d96-94a2-48cd-8d0e-431af8dced51 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.790364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.791348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.791631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.791911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] PATH_INFO: `/v3/projects/8cbe271df7c643d5b33e7696d1bbbf50/users/5a80f6b2508843f4b44e75fe8e343bcb/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.794069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.794510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:00.795290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.795290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.795543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] PATH_INFO: `/v3/projects/172381cd65a94b1fa2939d6af673f6ba/users/acbafaab99f848828c304ca5a45a55dc/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.810426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbe271df7c643d5b33e7696d1bbbf50, user_id=5a80f6b2508843f4b44e75fe8e343bcb, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.813158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.814223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172381cd65a94b1fa2939d6af673f6ba, user_id=acbafaab99f848828c304ca5a45a55dc, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.817142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.830681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 806/1601] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:00 2022] PUT /identity/v3/projects/8cbe271df7c643d5b33e7696d1bbbf50/users/5a80f6b2508843f4b44e75fe8e343bcb/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:00.832531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.847836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 796/1602] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:00 2022] PUT /identity/v3/projects/172381cd65a94b1fa2939d6af673f6ba/users/acbafaab99f848828c304ca5a45a55dc/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:00.847836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.914154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4141ad-52d2-44e8-967d-b93404e76e44 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.919816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.919974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21b9f2-dd43-4f95-907e-5bbe8c525551 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.920736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.921090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.921462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.922270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.924332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.924530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.925424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.925705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.925978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] PATH_INFO: `/v3/projects/a33a4f77f0ae4dc09d4f8c29babf3b03/users/6ca14b36fc754fe9b24abdfecf0f64c0/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.936430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 807/1603] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.937286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.947715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33a4f77f0ae4dc09d4f8c29babf3b03, user_id=6ca14b36fc754fe9b24abdfecf0f64c0, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.949669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e663bd-fe56-4d4b-8bde-83fd4acf7d44 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:00.951003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.954581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:00.955432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:00.955859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:00.956478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:00.958053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:00.960462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:00.968410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 797/1604] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:00 2022] PUT /identity/v3/projects/a33a4f77f0ae4dc09d4f8c29babf3b03/users/6ca14b36fc754fe9b24abdfecf0f64c0/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:00.973008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.977359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 808/1605] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:00.978731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:00.993855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6d632-af2e-4442-8dda-6cf44aa1b0f8 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.051043 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16aba1-e033-4a3d-bc3f-299337bac635 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.056633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807a068-c30e-4372-a66c-ad0907832151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.057775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807a068-c30e-4372-a66c-ad0907832151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.058252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807a068-c30e-4372-a66c-ad0907832151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.058697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807a068-c30e-4372-a66c-ad0907832151 admin admin] PATH_INFO: `/v3/projects/a2cc0415979f4460bff9ff14392f0986/users/81e888b7ff4746149eeea6861503b93a/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.076015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807a068-c30e-4372-a66c-ad0907832151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cc0415979f4460bff9ff14392f0986, user_id=81e888b7ff4746149eeea6861503b93a, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.076769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b3f91-6e23-4e72-9bf3-25f8f684a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.077601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3f91-6e23-4e72-9bf3-25f8f684a9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.077900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3f91-6e23-4e72-9bf3-25f8f684a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.078166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3f91-6e23-4e72-9bf3-25f8f684a9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.078933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b3f91-6e23-4e72-9bf3-25f8f684a9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.080545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b3f91-6e23-4e72-9bf3-25f8f684a9d9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.081302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807a068-c30e-4372-a66c-ad0907832151 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.095295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 809/1606] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:01.096991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ff7a6-8a31-41ca-b492-d0a9191f7bbc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.096991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ff7a6-8a31-41ca-b492-d0a9191f7bbc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.097259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ff7a6-8a31-41ca-b492-d0a9191f7bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.106306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 798/1607] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:00 2022] PUT /identity/v3/projects/a2cc0415979f4460bff9ff14392f0986/users/81e888b7ff4746149eeea6861503b93a/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:01.107534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.107946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.108317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.138229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:01.140135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:01.141748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-789ff7a6-8a31-41ca-b492-d0a9191f7bbc None None] MFA Rules not processed for user `5a80f6b2508843f4b44e75fe8e343bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:01.143995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-789ff7a6-8a31-41ca-b492-d0a9191f7bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:01.192927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:01.208056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:01.218889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 799/1608] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:01.218889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.230249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b0cde-fb87-461c-969a-258f6ab19320 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.231156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 810/1609] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:01.232510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.233082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.233082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.236643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.237657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.238036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.238408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] PATH_INFO: `/v3/projects/86b3133477f941cc8c46c2ad8b09cc8b/users/9d43cf357a624e119c442c108c2ac44a/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.255519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b3133477f941cc8c46c2ad8b09cc8b, user_id=9d43cf357a624e119c442c108c2ac44a, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.255775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:01.256842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:01.258458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.267454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 811/1610] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:01.268177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.279450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd63634-a941-4dfe-bacc-0a940784e195 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.284135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 800/1611] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:01 2022] PUT /identity/v3/projects/86b3133477f941cc8c46c2ad8b09cc8b/users/9d43cf357a624e119c442c108c2ac44a/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:01.284135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.348790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa752c-926c-49e0-88ab-9402d2ae5da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.349737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa752c-926c-49e0-88ab-9402d2ae5da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.350019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa752c-926c-49e0-88ab-9402d2ae5da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.350353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa752c-926c-49e0-88ab-9402d2ae5da7 admin admin] PATH_INFO: `/v3/projects/54baeb39b08b4d69aac8c4d42043112a/users/fe9a62b8fcdc400b9ceb993bb8f936e0/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.364874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4da9d-8e52-4556-b6d8-4a46a2595d2c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.370498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.371524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.371873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.372217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] PATH_INFO: `/v3/projects/8c3f2eb2f37545b2bad52d543340cb80/users/43de7ad82c2045119d184482ccd649f0/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.374888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa752c-926c-49e0-88ab-9402d2ae5da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54baeb39b08b4d69aac8c4d42043112a, user_id=fe9a62b8fcdc400b9ceb993bb8f936e0, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.377646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa752c-926c-49e0-88ab-9402d2ae5da7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.398549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 812/1612] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:01 2022] PUT /identity/v3/projects/54baeb39b08b4d69aac8c4d42043112a/users/fe9a62b8fcdc400b9ceb993bb8f936e0/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:01.404548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.404839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.405163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.405782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3f2eb2f37545b2bad52d543340cb80, user_id=43de7ad82c2045119d184482ccd649f0, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.410162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.430300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 801/1613] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:01 2022] PUT /identity/v3/projects/8c3f2eb2f37545b2bad52d543340cb80/users/43de7ad82c2045119d184482ccd649f0/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:01.431933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.445696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] MFA Rules not processed for user `81e888b7ff4746149eeea6861503b93a`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:01.448413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:01.566411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04aa20-a70d-43bf-8d4f-fd0b8336ca34 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.567898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 813/1614] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:01.568927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.574587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.575538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.575816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.576087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.660266 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:01.667962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.669999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.672835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22f94c-2ea3-4373-b0ca-20a4ce5347cf None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.680123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4534f-1028-4f1a-967b-1a86fab0e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.681328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4534f-1028-4f1a-967b-1a86fab0e446 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.681612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4534f-1028-4f1a-967b-1a86fab0e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.681971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4534f-1028-4f1a-967b-1a86fab0e446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.682614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 802/1615] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:01.683254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4534f-1028-4f1a-967b-1a86fab0e446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.683708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.686051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4534f-1028-4f1a-967b-1a86fab0e446 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.696767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146ea7-7537-4400-a498-6cc9f960605c service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.698904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:01.704210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b144a5-f14a-4310-a0a2-c27229ea6a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.705113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b144a5-f14a-4310-a0a2-c27229ea6a2b service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.705479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b144a5-f14a-4310-a0a2-c27229ea6a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.705820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b144a5-f14a-4310-a0a2-c27229ea6a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.719310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b144a5-f14a-4310-a0a2-c27229ea6a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.720877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b144a5-f14a-4310-a0a2-c27229ea6a2b service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.725384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 814/1616] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:01.727158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.727746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.728897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.737688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 803/1617] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:01.740570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.741046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.741461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.765975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:01.766301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:01.767505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:01.767728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:01.854916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 804/1618] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:01.855810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.865957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 815/1619] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:01.867165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.882924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c774107-d625-4dd8-8b89-4ab59951db35 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.883756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969e1d7-64fd-41e2-9eeb-5ae94b96e434 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.888799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.889485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.889759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.890037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.890818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.892444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.898355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.899079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.899454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.899739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.900712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.902669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.906876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 805/1620] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:01.908482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.925946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a2e07-93ad-4103-bcf3-01883f096df9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:01.933091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:01.934311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:01.934811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:01.935194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:01.936229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:01.939787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:01.940748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 816/1621] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:01.941598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:01.970724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 806/1622] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:01 2022] POST /identity/v3/users => generated 300 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:01.971801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.017064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855db61-c3da-4b39-9ddf-d08cdc55d725 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:02.021668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:02.023189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.023880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.024313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.068744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17258820-eacf-40fa-ac53-e015ec12dbc3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:02.080094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:02.080530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.080907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.081065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.154834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:02.156944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:02.171769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 817/1623] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:02.172787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.189056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc058d48-e700-4958-96d2-da736197fb16 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:02.208247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:02.209403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.209777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.210117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] PATH_INFO: `/v3/projects/8c3f2eb2f37545b2bad52d543340cb80/users/43de7ad82c2045119d184482ccd649f0/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.215473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3f2eb2f37545b2bad52d543340cb80, user_id=43de7ad82c2045119d184482ccd649f0, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:02.220583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:02.236511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:02.251661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 818/1624] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:02 2022] PUT /identity/v3/projects/8c3f2eb2f37545b2bad52d543340cb80/users/43de7ad82c2045119d184482ccd649f0/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:02.253556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.299023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:02.310684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:02.354884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 807/1625] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:02.355765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.371019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:02.380191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a4d56-3234-46ae-877a-3721ee32f210 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:02.497762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584ae24-2167-405c-85e0-7128c3a11b43 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:02.519976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:02.521262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.523138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.526767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.527941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:02.531459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:02.544842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8ac53-a69d-46e0-947e-bee9a5ab9d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:02.545802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ac53-a69d-46e0-947e-bee9a5ab9d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.546157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ac53-a69d-46e0-947e-bee9a5ab9d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.546504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ac53-a69d-46e0-947e-bee9a5ab9d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.549394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8ac53-a69d-46e0-947e-bee9a5ab9d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:02.550383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8ac53-a69d-46e0-947e-bee9a5ab9d76 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:02.589325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 808/1626] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:02.593013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.593812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.594552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.619603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 819/1627] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:02 2022] POST /identity/v3/users => generated 294 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:02.621134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.665574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d5bae-c20d-4851-b538-2f6701c6f9c3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:02.681288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:02.683347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:02.683861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:02.684405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] PATH_INFO: `/v3/projects/48dad1c624754b8fa57757b08d67ecd4/users/371b71d59027429b8131330b75eb3016/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:02.737148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] MFA Rules not processed for user `43de7ad82c2045119d184482ccd649f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:02.737148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:02.766823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48dad1c624754b8fa57757b08d67ecd4, user_id=371b71d59027429b8131330b75eb3016, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:02.784971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:02.847127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 820/1628] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:02 2022] PUT /identity/v3/projects/48dad1c624754b8fa57757b08d67ecd4/users/371b71d59027429b8131330b75eb3016/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:02.849532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.880927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 809/1629] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:02.881887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:02.911810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc280ea5-962b-4a9e-93b7-a02ac9b21552 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.111804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09ecf7-b20c-4db4-9a33-264e6bbfaa67 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.124454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.125400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.125683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.126075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.126901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:03.136090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:03.162452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 821/1630] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:03.163363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:03.219201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.220524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.221215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.223128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.224924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:03.228541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:03.249627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1f6a7-ddf2-434f-98a9-194401d2b064 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.253361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 810/1631] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:03.255373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:03.256633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.257315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.257597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.257872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.274994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:03.282532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6aead0-2254-40ab-8db8-d47a757461fd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.306101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.308208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.308663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.309148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.310581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:03.329620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:03.364917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 811/1632] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:03.366496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:03.388744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae464da-7172-424e-ba85-1df812cf0503 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.403502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:03.403758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf81060-8af7-456c-b9e7-95fc40c5f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.404431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf81060-8af7-456c-b9e7-95fc40c5f39c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.404712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf81060-8af7-456c-b9e7-95fc40c5f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.406391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:03.407229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf81060-8af7-456c-b9e7-95fc40c5f39c admin admin] PATH_INFO: `/v3/projects/48dad1c624754b8fa57757b08d67ecd4/users/371b71d59027429b8131330b75eb3016/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.414423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf81060-8af7-456c-b9e7-95fc40c5f39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48dad1c624754b8fa57757b08d67ecd4, user_id=371b71d59027429b8131330b75eb3016, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:03.418075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf81060-8af7-456c-b9e7-95fc40c5f39c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:03.430280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 822/1633] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:03.431551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:03.431972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:03.449260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 812/1634] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:03 2022] PUT /identity/v3/projects/48dad1c624754b8fa57757b08d67ecd4/users/371b71d59027429b8131330b75eb3016/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:03.452306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5c970-cf01-4335-a37c-98c6aa9770ee service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.460742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea88c8-a5a6-4509-94ae-d30fdd7569fe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.461028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea88c8-a5a6-4509-94ae-d30fdd7569fe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.465111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea88c8-a5a6-4509-94ae-d30fdd7569fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.546012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c1ea88c8-a5a6-4509-94ae-d30fdd7569fe None None] MFA Rules not processed for user `371b71d59027429b8131330b75eb3016`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:03.547875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c1ea88c8-a5a6-4509-94ae-d30fdd7569fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:03.578957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.581182 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.582101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.582545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] PATH_INFO: `/v3/projects/f8fe2ac64e77463f868efb0de979edcf/users/26936c2772df4930b84c6b886be78000/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.613840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fe2ac64e77463f868efb0de979edcf, user_id=26936c2772df4930b84c6b886be78000, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:03.618600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:03.647644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 823/1635] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:03 2022] PUT /identity/v3/projects/f8fe2ac64e77463f868efb0de979edcf/users/26936c2772df4930b84c6b886be78000/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:03.650969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:03.791041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 813/1636] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:03.794114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.794409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.794688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.829486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b1d6a-08a5-4fcd-a3b9-ea17f0c4cd3b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:03.854077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:03.855325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:03.861208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:03.861754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:03.891267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] MFA Rules not processed for user `26936c2772df4930b84c6b886be78000`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:03.900249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:04.067062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 814/1637] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:04.072867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.073142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:04.099335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e353-973b-4b05-b567-9cd6267198ac None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:04.100404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.103442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.110339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:04.111066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.112481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.112961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:04.131663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 824/1638] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:04.132495 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.260093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.263061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.310062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 815/1639] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:04.311403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09dce8-58b1-429d-b3ea-64db287c02a8 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:04.321957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:04.322911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.323516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.323956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:04.324839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.328225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.386782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.398642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 825/1640] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:04 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:04.404236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.406622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcc9b-f26e-4d17-96cd-2d2d9770c4b4 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:04.418404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:04.537392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ed314-136e-4eaa-8bb9-5a0a148aa698 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:04.552786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:04.553560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.553788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.554077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:04.554824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.571814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.583568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:04.586290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.587544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.587896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] PATH_INFO: `/v3/users/5a80f6b2508843f4b44e75fe8e343bcb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:04.593425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a80f6b2508843f4b44e75fe8e343bcb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.595463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:16:04.595463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) Jun 24 10:16:04.595463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: Jun 24 10:16:04.599713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.602857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 826/1641] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:04 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:16:04.609099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.623983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6ffe6-7557-48f1-9a97-475453d41a7d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:04.631171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:04.632271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.633026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.633752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:04.635560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.638261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.657910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 827/1642] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:16:04.664378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.689583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836412-a60b-4026-a06c-69bf14c096a3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:04.701254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:04.702349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.703242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.704416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] PATH_INFO: `/v3/projects/fb85430ae70247f084760e63fc517114/users/7879ac262ccd439b96fdaa6072412b15/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:04.737122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb85430ae70247f084760e63fc517114, user_id=7879ac262ccd439b96fdaa6072412b15, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:04.743260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:04.784783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 828/1643] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:04 2022] PUT /identity/v3/projects/fb85430ae70247f084760e63fc517114/users/7879ac262ccd439b96fdaa6072412b15/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:04.786529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:04.793534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a80f6b2508843f4b44e75fe8e343bcb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:04.888163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a80f6b2508843f4b44e75fe8e343bcb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:04.898342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:04.917896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a80f6b2508843f4b44e75fe8e343bcb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:04.947976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a80f6b2508843f4b44e75fe8e343bcb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:04.949634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d62caa5-6092-436b-bf35-59c0380b34a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a80f6b2508843f4b44e75fe8e343bcb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:04.969425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 816/1644] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:04 2022] DELETE /identity/v3/users/5a80f6b2508843f4b44e75fe8e343bcb => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:04.974048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:04.974390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:04.974970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:05.000873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f195b4-a08d-411d-95cd-6d79ace23c20 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:05.017383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:05.021201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:05.148141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:05.150512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:05.151064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:05.151410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:05.153690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:05.156380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:05.167046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 817/1645] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:05.172848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 829/1646] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:05.178931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:05.182742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:05.194835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:05.199179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:05.207686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5a46a-453c-4ffc-8169-0a3b2eaf6294 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:05.221419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfd85b-c64e-4d89-9021-67029881d2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:05.223183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd85b-c64e-4d89-9021-67029881d2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:05.223477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd85b-c64e-4d89-9021-67029881d2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:05.226017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd85b-c64e-4d89-9021-67029881d2c4 admin admin] PATH_INFO: `/v3/projects/fb85430ae70247f084760e63fc517114/users/7879ac262ccd439b96fdaa6072412b15/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:05.231237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfd85b-c64e-4d89-9021-67029881d2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb85430ae70247f084760e63fc517114, user_id=7879ac262ccd439b96fdaa6072412b15, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:05.241635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfd85b-c64e-4d89-9021-67029881d2c4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:05.284374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 818/1647] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:05 2022] PUT /identity/v3/projects/fb85430ae70247f084760e63fc517114/users/7879ac262ccd439b96fdaa6072412b15/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:05.311839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:05.313358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:05.313995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:05.356360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:05.360980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] MFA Rules not processed for user `7879ac262ccd439b96fdaa6072412b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:05.362378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:05.362824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c05a2-7508-472d-828b-e5fa3ee360a2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:05.495457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:05.498799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:05.502907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:05.502907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:05.503177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 819/1648] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:05.513343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:05.514819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:05.606478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:05.656838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:05.658798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:05.690143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 830/1649] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:05.695953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99455fe9-1594-4252-9b43-1aaa54556b9d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:05.705311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6f9d-4914-44d2-a92e-e1106dba339e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:05.706095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6f9d-4914-44d2-a92e-e1106dba339e service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:05.706424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6f9d-4914-44d2-a92e-e1106dba339e service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:05.706731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6f9d-4914-44d2-a92e-e1106dba339e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:05.732977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6f9d-4914-44d2-a92e-e1106dba339e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:05.736540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6f9d-4914-44d2-a92e-e1106dba339e service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:05.752372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 820/1650] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:06.612261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:06.614106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:06.629166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdd1c-4996-4834-a6b7-5a2b32752ff0 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:06.635473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:06.636212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:06.636500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:06.636781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:06.638725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:06.723666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:06.725556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:06.738121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 831/1651] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:06.814177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:06.814669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:06.814884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:06.879824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] MFA Rules not processed for user `9d43cf357a624e119c442c108c2ac44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:06.883865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:07.027610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 821/1652] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:07.031911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:07.033424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:07.063318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa64b3-e25e-419d-ae12-17acc1bd3e22 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:07.074643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:07.076224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:07.078972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:07.079293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:07.080065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:07.084477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:07.106385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:07.109674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:07.135335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d3d92-534c-442c-b9e8-a0bf33e2acaf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:07.153491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:07.156138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:07.156697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:07.157828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] PATH_INFO: `/v3/projects/8cbe271df7c643d5b33e7696d1bbbf50` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:07.174034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbe271df7c643d5b33e7696d1bbbf50)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:07.179240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 832/1653] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:07 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:07.180459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:07.182367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:07.275974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:07.332603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbe271df7c643d5b33e7696d1bbbf50', 'request_id': 'req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:07.350360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbe271df7c643d5b33e7696d1bbbf50', 'request_id': 'req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:07.371725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbe271df7c643d5b33e7696d1bbbf50', 'request_id': 'req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:07.377369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff6056-151d-4e8e-8569-e5cfe810710e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:07.418591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] The token cache is being invalidate because project 8cbe271df7c643d5b33e7696d1bbbf50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:07.420211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:07.423828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:07.437226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 822/1654] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:07 2022] DELETE /identity/v3/projects/8cbe271df7c643d5b33e7696d1bbbf50 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:07.437631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:07.439194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:07.668250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:07.677281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:07.680474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:07.682267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:07.689171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:07.692728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:07.760360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:07.830521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 833/1655] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:07 2022] POST /identity/v3/users => generated 308 bytes in 650 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:07.834605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:07.839869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:07.856126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ebb33-ec49-4a73-96b6-cb64938c4c00 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:07.875111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:07.877918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:07.881604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:07.882342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] PATH_INFO: `/v3/users/371b71d59027429b8131330b75eb3016` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:07.889866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371b71d59027429b8131330b75eb3016)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:07.899656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1980e7-f285-4c63-9c24-fc22aaf6e521 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:07.899789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:07.914682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:07.918409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:07.920064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:07.922005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:07.960414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:07.960414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:07.982616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 834/1656] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:07.991346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:08.030510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1278ea-234a-49ec-a642-6af7e7a89069 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:08.052042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:08.054060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:08.055267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:08.055911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] PATH_INFO: `/v3/projects/73c43ace79c842da946a40e236d5a151/users/ed2e0af1dd204bc9a9d8447766f3131d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:08.081544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371b71d59027429b8131330b75eb3016'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:08.100064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c43ace79c842da946a40e236d5a151, user_id=ed2e0af1dd204bc9a9d8447766f3131d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:08.104196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:08.120035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371b71d59027429b8131330b75eb3016'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:08.136914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371b71d59027429b8131330b75eb3016'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:08.162336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 835/1657] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:07 2022] PUT /identity/v3/projects/73c43ace79c842da946a40e236d5a151/users/ed2e0af1dd204bc9a9d8447766f3131d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:08.165816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:08.169740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371b71d59027429b8131330b75eb3016'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:08.171215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371b71d59027429b8131330b75eb3016'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:08.189051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 823/1658] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:07 2022] DELETE /identity/v3/users/371b71d59027429b8131330b75eb3016 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:08.255102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4655cbe-9d2d-4591-89de-f32a361f07a2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:08.402871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:08.404945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:08.405334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:08.405957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:08.407917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:08.409676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:08.468650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 836/1659] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:08.472981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:08.480273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:08.505625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf955f-a2a5-4632-ad83-1e3e10f1f130 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:08.517207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:08.518217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:08.518463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:08.518761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] PATH_INFO: `/v3/projects/73c43ace79c842da946a40e236d5a151/users/ed2e0af1dd204bc9a9d8447766f3131d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:08.524746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c43ace79c842da946a40e236d5a151, user_id=ed2e0af1dd204bc9a9d8447766f3131d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:08.527497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:08.558843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 824/1660] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:08 2022] PUT /identity/v3/projects/73c43ace79c842da946a40e236d5a151/users/ed2e0af1dd204bc9a9d8447766f3131d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:09.828691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:09.831534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:09.865462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcad8f-4644-4110-9dd4-eb8fff4f73a0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:10.055200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:10.058589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.059126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.059410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] PATH_INFO: `/v3/projects/48dad1c624754b8fa57757b08d67ecd4` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.067762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48dad1c624754b8fa57757b08d67ecd4)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:10.068994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: WARNING py.warnings [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 10:16:10.068994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: warnings.warn(msg) Jun 24 10:16:10.068994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: Jun 24 10:16:10.070438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:10.075222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:10.276084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48dad1c624754b8fa57757b08d67ecd4', 'request_id': 'req-c4e667ea-458f-4b54-abba-5fa608aee726'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.388316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:10.398724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:10.417451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48dad1c624754b8fa57757b08d67ecd4', 'request_id': 'req-c4e667ea-458f-4b54-abba-5fa608aee726'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.439766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55ec4-3ac3-4801-a9a8-efe29b73eb40 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:10.450469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48dad1c624754b8fa57757b08d67ecd4', 'request_id': 'req-c4e667ea-458f-4b54-abba-5fa608aee726'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.491554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:10.522897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] The token cache is being invalidate because project 48dad1c624754b8fa57757b08d67ecd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:10.525599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.528490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c4e667ea-458f-4b54-abba-5fa608aee726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.532197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 837/1661] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:09 2022] DELETE /identity/v3/projects/48dad1c624754b8fa57757b08d67ecd4 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:10.540958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358407e-85d1-40c3-b6d0-908629295b7f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.547764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358407e-85d1-40c3-b6d0-908629295b7f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.547868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358407e-85d1-40c3-b6d0-908629295b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.632145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0358407e-85d1-40c3-b6d0-908629295b7f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:10.634094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0358407e-85d1-40c3-b6d0-908629295b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:10.659814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:10.661529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.662003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.662305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] PATH_INFO: `/v3/users/54c73c484dcc4b5686a9226979ad8fc9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.663779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c73c484dcc4b5686a9226979ad8fc9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:10.666344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:10.676545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 838/1662] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:10.692484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.692668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.692952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.728826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:10.730252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:10.751499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 839/1663] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:10.755854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:10.781135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3603e2-d180-44b0-af35-266bc4962b82 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:10.798628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c73c484dcc4b5686a9226979ad8fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.800577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:10.801596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.802876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.805285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.806400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:10.818669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:10.825027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:10.851822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c73c484dcc4b5686a9226979ad8fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.874860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 840/1664] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:10.877400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:10.886671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c73c484dcc4b5686a9226979ad8fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.898471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c73c484dcc4b5686a9226979ad8fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.902791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e3076cf4-c004-4393-8a25-9f5525eb85e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c73c484dcc4b5686a9226979ad8fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:10.913680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5e825-dc43-4ba1-96e7-719727855247 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:10.926383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 825/1665] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:10 2022] DELETE /identity/v3/users/54c73c484dcc4b5686a9226979ad8fc9 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:10.931083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.931341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.934943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.938401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:10.939430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:10.940496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:10.944118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:10.946031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:10.948959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:10.999932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:11.004077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:11.038189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 841/1666] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:10 2022] POST /identity/v3/projects => generated 401 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:11.039292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:11.043013 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:11.181834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 826/1667] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:11.201482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f29b7-b822-4e3c-9769-58f940b35a66 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:11.214562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:11.215321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:11.215607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:11.215978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:11.217538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:11.220274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:11.221793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:11.228663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:11.279760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 842/1668] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:11 2022] POST /identity/v3/users => generated 301 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:11.284457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:11.307260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf64af-5691-4bcb-88ea-1412ea6c7573 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:11.326982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:11.327695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:11.328404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:11.328697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:11.328978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:11.330694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:11.336374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:11.370478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cd41-0544-4a8c-ac4a-ad2a3cbc11f8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:11.371399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 843/1669] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:11.372459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:11.398431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0ed74-144d-4d77-a0cb-da29a08e38b0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:11.412467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:11.413276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:11.416136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:11.416593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:11.417415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:11.421315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:11.443300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 844/1670] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:11.452065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:11.482047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b763cd-e0a8-4a7d-91f7-e5209565a1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:11.483625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b763cd-e0a8-4a7d-91f7-e5209565a1ad service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:11.484129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b763cd-e0a8-4a7d-91f7-e5209565a1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:11.484616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b763cd-e0a8-4a7d-91f7-e5209565a1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:11.493795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffe6b0-9ae5-47b2-a007-0190fa345f40 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:11.502807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:11.504851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:11.505389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:11.505874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] PATH_INFO: `/v3/projects/79651536f6be44d089cb6ba862ef76a4/users/c135d652a1534de8966d2bd7f12799ec/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:11.508078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b763cd-e0a8-4a7d-91f7-e5209565a1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:11.512813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b763cd-e0a8-4a7d-91f7-e5209565a1ad service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:11.513720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:11.539418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 827/1671] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:11.541475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79651536f6be44d089cb6ba862ef76a4, user_id=c135d652a1534de8966d2bd7f12799ec, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:11.545980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:11.593474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:11.594003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:11.594478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:11.596257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 845/1672] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:11 2022] PUT /identity/v3/projects/79651536f6be44d089cb6ba862ef76a4/users/c135d652a1534de8966d2bd7f12799ec/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:11.646681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] MFA Rules not processed for user `c135d652a1534de8966d2bd7f12799ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:11.648368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:11.870808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 828/1673] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:11.879044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:12.051567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5a638-a944-4435-bdef-796f239b89d9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:12.064880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:12.065603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:12.065960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:12.066304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:12.067116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:12.069179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:12.148439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 846/1674] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:11 2022] POST /identity/v3/projects => generated 401 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:12.152420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:12.172349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa6de-cf3b-4901-896f-5f7ea1d0891c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:12.350211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:12.351506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:12.351929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:12.352535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:12.358625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:12.371401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:12.446125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 829/1675] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:12 2022] POST /identity/v3/users => generated 307 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:12.450127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:12.474239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20512b5-1b61-49ac-9d75-8a9b2df94b52 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:12.505595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:12.506496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:12.506904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:12.508065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:12.509638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:12.516166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:12.567488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 847/1676] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:16:12.571519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:12.592734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707028f1-c0ce-46c8-a500-6cf5e31f9533 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:12.617410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:12.619371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:12.619476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:12.619830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] PATH_INFO: `/v3/projects/709010bf0d694d9fa40b122230322fe3/users/68d2caca05684ad2abdb274b5f0be305/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:12.655154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=709010bf0d694d9fa40b122230322fe3, user_id=68d2caca05684ad2abdb274b5f0be305, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:12.661300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:12.672500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:12.705056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 830/1677] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:12 2022] PUT /identity/v3/projects/709010bf0d694d9fa40b122230322fe3/users/68d2caca05684ad2abdb274b5f0be305/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:12.710313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:12.726111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193dbf-742a-4c0f-b93a-0f0a603b9e18 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:12.931296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:12.932148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:12.932529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:12.932895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:12.933752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:12.935764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:12.992901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:13.003890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 848/1678] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:13.006631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6605a-30b9-414b-b150-271e244e01cc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:13.033894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f308df-0663-40eb-bab6-5b9d71d8d600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:13.034983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f308df-0663-40eb-bab6-5b9d71d8d600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:13.035371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f308df-0663-40eb-bab6-5b9d71d8d600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:13.035746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f308df-0663-40eb-bab6-5b9d71d8d600 admin admin] PATH_INFO: `/v3/projects/709010bf0d694d9fa40b122230322fe3/users/68d2caca05684ad2abdb274b5f0be305/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:13.042182 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:13.044244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:13.047776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f308df-0663-40eb-bab6-5b9d71d8d600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=709010bf0d694d9fa40b122230322fe3, user_id=68d2caca05684ad2abdb274b5f0be305, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:13.058106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f308df-0663-40eb-bab6-5b9d71d8d600 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:13.072329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0465a-c0df-4ecd-8791-eb0de5041274 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:13.098901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:13.102195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:13.103459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:13.103669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 831/1679] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:12 2022] PUT /identity/v3/projects/709010bf0d694d9fa40b122230322fe3/users/68d2caca05684ad2abdb274b5f0be305/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:13.104149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] PATH_INFO: `/v3/projects/1f1538c924054dc694f1207737d55dda` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:13.111491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1538c924054dc694f1207737d55dda)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:13.114899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:13.115310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:13.116214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:13.119398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:13.222590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] MFA Rules not processed for user `68d2caca05684ad2abdb274b5f0be305`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:13.224703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:13.291292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1538c924054dc694f1207737d55dda', 'request_id': 'req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:13.293764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:13.309985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1538c924054dc694f1207737d55dda', 'request_id': 'req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:13.335172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1538c924054dc694f1207737d55dda', 'request_id': 'req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:13.359344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] The token cache is being invalidate because project 1f1538c924054dc694f1207737d55dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:13.360702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:13.361889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b65e1e81-0e5b-4f85-97dd-28ee8d82c83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:13.365870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 849/1680] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:13 2022] DELETE /identity/v3/projects/1f1538c924054dc694f1207737d55dda => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:13.369903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:13.371000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:13.372767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:13.412984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 832/1681] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:13.414241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:13.419631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:13.456455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] MFA Rules not processed for user `ed2e0af1dd204bc9a9d8447766f3131d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:13.461327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:13.595783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e447e8-054b-4f7f-a2da-7faabdb7e2ae None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:13.667129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 850/1682] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:13.668576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:13.672769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:13.806481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:13.808992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:13.809865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:13.810785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] PATH_INFO: `/v3/users/43de7ad82c2045119d184482ccd649f0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:13.813404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43de7ad82c2045119d184482ccd649f0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:13.816836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:13.820047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92616b9-7bf6-4bbe-823f-ed3ea1e2cf1f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:13.832926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:13.833631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:13.833904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:13.834185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:13.853811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:13.992053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:13.993638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:14.016355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 851/1683] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:13 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:14.042767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43de7ad82c2045119d184482ccd649f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:14.150657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43de7ad82c2045119d184482ccd649f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:14.183536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43de7ad82c2045119d184482ccd649f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:14.228836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43de7ad82c2045119d184482ccd649f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:14.233234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43de7ad82c2045119d184482ccd649f0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:14.254153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:14.256142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:14.266160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 833/1684] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:13 2022] DELETE /identity/v3/users/43de7ad82c2045119d184482ccd649f0 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:14.283096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9571160-c24c-4400-83ee-d4ac4c192ddc service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:14.295740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:14.297007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:14.312831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:14.425860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:14.427446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:14.427850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:14.428213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:14.430771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:14.434820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d769056-1bfd-4907-8b1d-eeeb6efba72c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:14.445897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:14.446823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:14.447295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:14.447567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:14.460354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:14.639116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:14.648724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:14.659797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:14.663888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabbec5-bc0a-4146-8729-a7450217ff3d service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:14.680134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 852/1685] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:14.688933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 834/1686] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:15.429417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:15.431296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:15.555050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226bfd4-ca05-4feb-9c77-f4d7b939e781 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:15.555401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226bfd4-ca05-4feb-9c77-f4d7b939e781 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:15.555694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226bfd4-ca05-4feb-9c77-f4d7b939e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:15.649153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78409-a281-43cc-9a9a-8ec30a10275d service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:15.649488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e226bfd4-ca05-4feb-9c77-f4d7b939e781 None None] MFA Rules not processed for user `0a66b13572904a309d3180afc7278070`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:15.651052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e226bfd4-ca05-4feb-9c77-f4d7b939e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:15.678303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:15.682592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:15.685847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:15.686358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] PATH_INFO: `/v3/projects/8c3f2eb2f37545b2bad52d543340cb80` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:15.693280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3f2eb2f37545b2bad52d543340cb80)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:15.705315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:15.761538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:15.844938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3f2eb2f37545b2bad52d543340cb80', 'request_id': 'req-03cc4ba1-26f4-45e8-893b-8bb460bfc361'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:15.862070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 835/1687] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:15.884915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3f2eb2f37545b2bad52d543340cb80', 'request_id': 'req-03cc4ba1-26f4-45e8-893b-8bb460bfc361'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:15.931289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3f2eb2f37545b2bad52d543340cb80', 'request_id': 'req-03cc4ba1-26f4-45e8-893b-8bb460bfc361'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:15.934545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88b240-c9fc-4d9a-acef-677da9fa4452 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:15.935250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88b240-c9fc-4d9a-acef-677da9fa4452 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:15.935640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88b240-c9fc-4d9a-acef-677da9fa4452 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:15.937552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 836/1688] 10.222.0.64 () {54 vars in 961 bytes} [Fri Jun 24 10:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:15.951218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:15.951659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:15.952050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:15.960576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:15.971471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] The token cache is being invalidate because project 8c3f2eb2f37545b2bad52d543340cb80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:15.973243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:15.976488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-03cc4ba1-26f4-45e8-893b-8bb460bfc361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:15.992299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 853/1689] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:15 2022] DELETE /identity/v3/projects/8c3f2eb2f37545b2bad52d543340cb80 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:15.997878 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:15.998378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:15.999906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:16.008073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:16.009919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:16.100113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:16.117246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:16.150255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 837/1690] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Jun 24 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:16.151492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:16.168529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ccc1f-9691-484f-87a6-3e834c02da33 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:16.177146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:16.177911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:16.178209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:16.178488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:16.181231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:16.304430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:16.306916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed5b0-6b4d-482d-b186-dedbf26051cb service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:16.332270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 838/1691] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:16.409357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 854/1692] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:16.417295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:16.417583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:16.417869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:16.455011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:16.456270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:16.481337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 839/1693] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:16.485438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:16.498777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c26c5-7de1-4e30-8bc1-1d991c0ab6ff None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:16.509857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:16.512177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:16.512646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:16.513071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:16.514733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:16.515492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:16.523170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:16.541335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 855/1694] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:16.545684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:16.561569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313a3a3-9d81-4f7f-b83f-99daa72c4d0d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:16.623606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:16.624295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:16.624592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:16.625064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:16.626090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:16.629725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:16.685533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 840/1695] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:16 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:16.697313 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:16.723536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263c0d8-f43c-453b-859a-8ca8e5e3e02e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:16.908218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:16.909126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:16.909562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:16.909995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:16.910908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:16.913545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:16.965663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 856/1696] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:16 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:16.970993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:16.984835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1f86d-366a-4184-af6c-ecb89a9e06fb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.014988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.019218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.021719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.022007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.023435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.028129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:17.064899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 841/1697] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:17.070733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:17.093896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500306b-a550-4e38-991c-638d6a8f016e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.115525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5872486-8a2c-44e9-a2a2-e907ffd3080e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.116426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872486-8a2c-44e9-a2a2-e907ffd3080e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.116850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872486-8a2c-44e9-a2a2-e907ffd3080e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.117296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5872486-8a2c-44e9-a2a2-e907ffd3080e admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.118124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5872486-8a2c-44e9-a2a2-e907ffd3080e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.119843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5872486-8a2c-44e9-a2a2-e907ffd3080e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:17.155971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 857/1698] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:17.159695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:17.175854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422b8db-8085-41ad-a780-4968bc92f7cf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.183422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.184486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.184776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.185055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] PATH_INFO: `/v3/projects/3ee36d9024964e9c89239970cd5edda9/users/b3226078fddc4d3390bd2a76639890bc/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.226179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee36d9024964e9c89239970cd5edda9, user_id=b3226078fddc4d3390bd2a76639890bc, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.239079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:17.281974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 842/1699] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:17 2022] PUT /identity/v3/projects/3ee36d9024964e9c89239970cd5edda9/users/b3226078fddc4d3390bd2a76639890bc/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:17.289189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.289533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.289869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.357713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] MFA Rules not processed for user `b3226078fddc4d3390bd2a76639890bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:17.360646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:17.492081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:17.508190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 858/1700] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:17.509673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:17.515040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:17.648400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aea126-c06e-48ea-ae54-701ddf561254 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.661714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.662439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.662670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.663515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.664145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.666144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:17.717654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 843/1701] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:17 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:17.722085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:17.736399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ac00b-d992-46ac-aab7-9e28426b5cfe None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.855943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.857202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.857637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.858053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.858946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.864914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:17.902399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 859/1702] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:17 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:17.904065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:17.915358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8273e65-dbeb-4692-a22a-900dd0506e57 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.920933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72edd07-cbd0-4bea-bc66-d4838ff43f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.921740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72edd07-cbd0-4bea-bc66-d4838ff43f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.922034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72edd07-cbd0-4bea-bc66-d4838ff43f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.922314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72edd07-cbd0-4bea-bc66-d4838ff43f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.923205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72edd07-cbd0-4bea-bc66-d4838ff43f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.925518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72edd07-cbd0-4bea-bc66-d4838ff43f0e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:17.944961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 844/1703] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:16:17.946637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:17.961606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5ffd0-d587-41c4-9955-28979abc5e1e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:17.969899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06966375-9553-416a-b254-6f8f6efcf21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:17.971764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06966375-9553-416a-b254-6f8f6efcf21a admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.972263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06966375-9553-416a-b254-6f8f6efcf21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.972808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06966375-9553-416a-b254-6f8f6efcf21a admin admin] PATH_INFO: `/v3/projects/c334aeb0d6fe499b9caf09111c6f81f5/users/8247e3b9b8a54a919bf1550ad39b1646/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.996055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e46bc7-4e48-4694-a13c-12cab88dcf0d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:17.996055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e46bc7-4e48-4694-a13c-12cab88dcf0d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:17.996707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06966375-9553-416a-b254-6f8f6efcf21a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c334aeb0d6fe499b9caf09111c6f81f5, user_id=8247e3b9b8a54a919bf1550ad39b1646, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:17.996814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e46bc7-4e48-4694-a13c-12cab88dcf0d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:17.997559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 845/1704] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:16:17.999442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06966375-9553-416a-b254-6f8f6efcf21a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:18.026478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 860/1705] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:17 2022] PUT /identity/v3/projects/c334aeb0d6fe499b9caf09111c6f81f5/users/8247e3b9b8a54a919bf1550ad39b1646/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:18.030791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e46bc7-4e48-4694-a13c-12cab88dcf0d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:18.098284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7857902-5e07-4ad1-b3bb-747f9b90edbd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.098747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7857902-5e07-4ad1-b3bb-747f9b90edbd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.098860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7857902-5e07-4ad1-b3bb-747f9b90edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.133612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d7857902-5e07-4ad1-b3bb-747f9b90edbd None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:18.134985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d7857902-5e07-4ad1-b3bb-747f9b90edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:18.172923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e46bc7-4e48-4694-a13c-12cab88dcf0d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:18.178409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:18.179149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.179430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.179705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.180529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:18.192741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:18.227771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 846/1706] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:18.232694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:18.253599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9de18-0c82-4532-8d34-938aea0ac17f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:18.267887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26b044-0d9b-408f-997a-81711970225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:18.269567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26b044-0d9b-408f-997a-81711970225e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.269718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26b044-0d9b-408f-997a-81711970225e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.270665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26b044-0d9b-408f-997a-81711970225e admin admin] PATH_INFO: `/v3/projects/c334aeb0d6fe499b9caf09111c6f81f5/users/8247e3b9b8a54a919bf1550ad39b1646/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.277091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26b044-0d9b-408f-997a-81711970225e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c334aeb0d6fe499b9caf09111c6f81f5, user_id=8247e3b9b8a54a919bf1550ad39b1646, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:18.287283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26b044-0d9b-408f-997a-81711970225e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:18.369160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 847/1707] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:18 2022] PUT /identity/v3/projects/c334aeb0d6fe499b9caf09111c6f81f5/users/8247e3b9b8a54a919bf1550ad39b1646/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:18.374227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.376336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.377543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.443100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 861/1708] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:18.490193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] MFA Rules not processed for user `8247e3b9b8a54a919bf1550ad39b1646`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:18.491418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:18.512090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860a01a-b267-4aad-92c0-e2b8ee025f0b None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.512553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860a01a-b267-4aad-92c0-e2b8ee025f0b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.512642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860a01a-b267-4aad-92c0-e2b8ee025f0b None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.514673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 862/1709] 10.222.0.64 () {54 vars in 960 bytes} [Fri Jun 24 10:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:18.549495 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.550041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.550473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.609076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:18.615323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:18.663895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:18.735547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 848/1710] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:18.791331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:18.813228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:18.814569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:18.836623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 863/1711] 10.222.0.64 () {58 vars in 1057 bytes} [Fri Jun 24 10:16:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:18.847491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:18.863506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8defe7-8a27-4e6e-80b3-e04c66424cde None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:18.870621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:18.872057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.872511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.873054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.886010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:18.888714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:18.911071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 864/1712] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:18.955135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c7393-f602-49ce-a95d-d77e7128c546 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:18.964042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:18.964720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:18.965008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:18.965285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:18.986445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:18.987979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:19.028539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 849/1713] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:19.217784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:19.349809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:19.351313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:19.385088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afc0a-6fa0-4b68-8438-eb0e3302ecc2 service placement] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:19.392538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:19.394263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:19.395150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:19.395334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] PATH_INFO: `/v3/users/b3226078fddc4d3390bd2a76639890bc` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:19.397221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3226078fddc4d3390bd2a76639890bc)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:19.403232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:19.509914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3226078fddc4d3390bd2a76639890bc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.551819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3226078fddc4d3390bd2a76639890bc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.562782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537836-edab-466e-a7fb-e2a5ad1f5115 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:19.572950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:19.573646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3226078fddc4d3390bd2a76639890bc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.580521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:19.582218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:19.584409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] PATH_INFO: `/v3/users/81e888b7ff4746149eeea6861503b93a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:19.587498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e888b7ff4746149eeea6861503b93a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:19.590879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3226078fddc4d3390bd2a76639890bc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.593366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:19.594963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3226078fddc4d3390bd2a76639890bc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.619450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 865/1714] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:19 2022] DELETE /identity/v3/users/b3226078fddc4d3390bd2a76639890bc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:19.622630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:19.627027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:19.727698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e888b7ff4746149eeea6861503b93a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.763121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e888b7ff4746149eeea6861503b93a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.797280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:19.799872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e888b7ff4746149eeea6861503b93a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.814824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc79e-91e6-466d-bbe9-f6149ff311d8 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:19.815599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e888b7ff4746149eeea6861503b93a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.819634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fc509a63-01c5-412e-9cba-6349f29eec0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e888b7ff4746149eeea6861503b93a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:19.831267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 850/1715] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:19 2022] DELETE /identity/v3/users/81e888b7ff4746149eeea6861503b93a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:19.834162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:19.834162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:19.834162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:19.834162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] PATH_INFO: `/v3/users/c135d652a1534de8966d2bd7f12799ec` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:19.835866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520f13d-10eb-4e71-b712-202dfd5b3862 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:19.837403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520f13d-10eb-4e71-b712-202dfd5b3862 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:19.837753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520f13d-10eb-4e71-b712-202dfd5b3862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:19.839773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c135d652a1534de8966d2bd7f12799ec)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:19.841624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:19.887936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:19.928924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7520f13d-10eb-4e71-b712-202dfd5b3862 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:19.935657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7520f13d-10eb-4e71-b712-202dfd5b3862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:20.080048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c135d652a1534de8966d2bd7f12799ec'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:20.139372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c135d652a1534de8966d2bd7f12799ec'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:20.169375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c135d652a1534de8966d2bd7f12799ec'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:20.192425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c135d652a1534de8966d2bd7f12799ec'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:20.197810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-53461a2c-8202-44de-b206-bb0a59c803a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c135d652a1534de8966d2bd7f12799ec'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:20.223374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 851/1716] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:20.232924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:20.235205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 866/1717] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:19 2022] DELETE /identity/v3/users/c135d652a1534de8966d2bd7f12799ec => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:20.236100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:20.236610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:20.247518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258fe0c-d059-4a28-8fee-382af32c484b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:20.247814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258fe0c-d059-4a28-8fee-382af32c484b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:20.248098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258fe0c-d059-4a28-8fee-382af32c484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:20.284615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:20.288696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:20.297124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b258fe0c-d059-4a28-8fee-382af32c484b None None] MFA Rules not processed for user `0a66b13572904a309d3180afc7278070`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:20.299709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b258fe0c-d059-4a28-8fee-382af32c484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:20.443195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 867/1718] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:20.444603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:20.444890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:20.445170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:20.483521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:20.486961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:20.487125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 852/1719] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:20.487599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:20.492346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:20.510716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 868/1720] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:20.522874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:20.530131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:20.673823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23471072-0829-4e4f-b79c-b973ffc6f77e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:20.682782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:20.685264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:20.688137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:20.688137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:20.742462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06185e20-bbca-4b12-8e89-54bc6a256020 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:20.748990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:20.752721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:20.753535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9530aa-4d4c-43e3-afce-5cdb7653cffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:20.754229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9530aa-4d4c-43e3-afce-5cdb7653cffe service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:20.754515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9530aa-4d4c-43e3-afce-5cdb7653cffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:20.754814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9530aa-4d4c-43e3-afce-5cdb7653cffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:20.776630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9530aa-4d4c-43e3-afce-5cdb7653cffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:20.785897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9530aa-4d4c-43e3-afce-5cdb7653cffe service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:20.806911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 853/1721] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:20.810185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:20.810367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 869/1722] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:20.828006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3627f-d648-4466-bffa-911eccaea2a0 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:20.840254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4568043-c505-47d9-9882-e7d5aad686e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:20.841090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4568043-c505-47d9-9882-e7d5aad686e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:20.841380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4568043-c505-47d9-9882-e7d5aad686e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:20.842116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4568043-c505-47d9-9882-e7d5aad686e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:20.870313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4568043-c505-47d9-9882-e7d5aad686e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:20.871019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4568043-c505-47d9-9882-e7d5aad686e3 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:20.890894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 854/1723] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:21.030015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae851a5e-9601-4054-be44-3e285c4c1adb None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.030536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae851a5e-9601-4054-be44-3e285c4c1adb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.031174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae851a5e-9601-4054-be44-3e285c4c1adb None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.032868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 870/1724] 10.222.0.64 () {54 vars in 961 bytes} [Fri Jun 24 10:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:21.040591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a332-19fe-45e0-a724-dad845e567a6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.040877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a332-19fe-45e0-a724-dad845e567a6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.041231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a332-19fe-45e0-a724-dad845e567a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.075336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5476a332-19fe-45e0-a724-dad845e567a6 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:21.076578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5476a332-19fe-45e0-a724-dad845e567a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.144982 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:21.163090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161514-8537-426c-94ee-30113263ab55 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.163433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161514-8537-426c-94ee-30113263ab55 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.163849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161514-8537-426c-94ee-30113263ab55 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.165829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 871/1725] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:21.190940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206813b0-d591-4b0b-918e-a605deef5d82 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.191539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206813b0-d591-4b0b-918e-a605deef5d82 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.191953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206813b0-d591-4b0b-918e-a605deef5d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.264103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-206813b0-d591-4b0b-918e-a605deef5d82 None None] MFA Rules not processed for user `edd302416f834b5b9b2f1a8c9955349d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:21.267905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-206813b0-d591-4b0b-918e-a605deef5d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.282905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 855/1726] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Jun 24 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:21.283496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f5539-3018-4e65-bf3d-6dfbca81f73d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.283744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f5539-3018-4e65-bf3d-6dfbca81f73d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.286033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f5539-3018-4e65-bf3d-6dfbca81f73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.310672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:21.371816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-494f5539-3018-4e65-bf3d-6dfbca81f73d None None] MFA Rules not processed for user `6ca14b36fc754fe9b24abdfecf0f64c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:21.373449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-494f5539-3018-4e65-bf3d-6dfbca81f73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.466060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 872/1727] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:21.477183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aaed3-a810-4853-b410-43274801e81f None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.478785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aaed3-a810-4853-b410-43274801e81f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.479966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aaed3-a810-4853-b410-43274801e81f None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.485352 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 873/1728] 10.222.0.64 () {54 vars in 912 bytes} [Fri Jun 24 10:16:21 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:21.487414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aaed3-a810-4853-b410-43274801e81f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:21.493332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d77aaed3-a810-4853-b410-43274801e81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.521488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aaed3-a810-4853-b410-43274801e81f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:21.527004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:21.528062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 856/1729] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:21.528297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.528682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.529073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] PATH_INFO: `/v3/users/9d43cf357a624e119c442c108c2ac44a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.530245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.530542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.531716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.535983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d43cf357a624e119c442c108c2ac44a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:21.538078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:21.555840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:21.557543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.573183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 857/1730] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:21.574965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:21.577164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.600867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfd7b4-85b4-4e9b-8770-7cf71072bda3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:21.607165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:21.608044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.608426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.608794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.619477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:21.621791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:21.629116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d43cf357a624e119c442c108c2ac44a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:21.633977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 858/1731] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:21.634990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:21.636616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.646592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d43cf357a624e119c442c108c2ac44a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:21.658464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d819f8-e3ae-4df3-8bee-9e1f8413f871 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:21.673416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d43cf357a624e119c442c108c2ac44a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:21.679747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:21.680808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.681653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.682771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.685491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d43cf357a624e119c442c108c2ac44a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:21.686365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d43cf357a624e119c442c108c2ac44a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:21.695629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:21.697767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:21.709150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 859/1732] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:21.709857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 874/1733] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:21 2022] DELETE /identity/v3/users/9d43cf357a624e119c442c108c2ac44a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:21.711032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:21.713471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:21.828420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0399b2-6fe8-4166-8376-8cfe2782bd87 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:21.919371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:21.936033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:21.937507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:21.937896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:21.938427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:21.942969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:22.141854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:22.144673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:22.160050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 860/1734] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:22.473960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:22.476266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:22.484651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:22.486294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:22.625306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88420a50-1b52-4cef-aeb3-743075609c81 service glance] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:22.643445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:22.645327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:22.647599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:22.650022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] PATH_INFO: `/v3/projects/79651536f6be44d089cb6ba862ef76a4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:22.652387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=79651536f6be44d089cb6ba862ef76a4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:22.657533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:22.725555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcee88a-d63d-4cbf-bf61-8e0d491bcab8 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:22.744889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:22.746098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:22.746587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:22.746839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] PATH_INFO: `/v3/projects/a2cc0415979f4460bff9ff14392f0986` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:22.748431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cc0415979f4460bff9ff14392f0986)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:22.751050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:22.837429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79651536f6be44d089cb6ba862ef76a4', 'request_id': 'req-954047f4-9462-46ba-9a94-6194885ab4db'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:22.861412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79651536f6be44d089cb6ba862ef76a4', 'request_id': 'req-954047f4-9462-46ba-9a94-6194885ab4db'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:22.887175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79651536f6be44d089cb6ba862ef76a4', 'request_id': 'req-954047f4-9462-46ba-9a94-6194885ab4db'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:22.929033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:22.944532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] The token cache is being invalidate because project 79651536f6be44d089cb6ba862ef76a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:22.944864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:22.947848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:22.954861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 861/1735] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:22 2022] DELETE /identity/v3/projects/79651536f6be44d089cb6ba862ef76a4 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:22.954861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:22.958436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:23.006308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cc0415979f4460bff9ff14392f0986', 'request_id': 'req-1e1da48f-a6ea-40d4-a787-720262762978'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.016421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:23.018509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cc0415979f4460bff9ff14392f0986', 'request_id': 'req-1e1da48f-a6ea-40d4-a787-720262762978'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.069593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cc0415979f4460bff9ff14392f0986', 'request_id': 'req-1e1da48f-a6ea-40d4-a787-720262762978'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.137323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] The token cache is being invalidate because project a2cc0415979f4460bff9ff14392f0986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:23.138672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.142798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.151858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 875/1736] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:22 2022] DELETE /identity/v3/projects/a2cc0415979f4460bff9ff14392f0986 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:23.158755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:23.168672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:23.194028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954047f4-9462-46ba-9a94-6194885ab4db admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:23.320963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:23.322804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:23.323223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:23.323370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] PATH_INFO: `/v3/projects/86b3133477f941cc8c46c2ad8b09cc8b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:23.331265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b3133477f941cc8c46c2ad8b09cc8b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:23.333559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:23.388118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1da48f-a6ea-40d4-a787-720262762978 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:23.397956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:23.399982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:23.400647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:23.401376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] PATH_INFO: `/v3/projects/3ee36d9024964e9c89239970cd5edda9` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:23.404462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee36d9024964e9c89239970cd5edda9)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:23.408434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:23.455424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b3133477f941cc8c46c2ad8b09cc8b', 'request_id': 'req-cb2a5e87-ca5a-4092-bae0-f503b1618171'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.468847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b3133477f941cc8c46c2ad8b09cc8b', 'request_id': 'req-cb2a5e87-ca5a-4092-bae0-f503b1618171'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.483966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b3133477f941cc8c46c2ad8b09cc8b', 'request_id': 'req-cb2a5e87-ca5a-4092-bae0-f503b1618171'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.508752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] The token cache is being invalidate because project 86b3133477f941cc8c46c2ad8b09cc8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:23.509939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.511898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.515745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 862/1737] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:22 2022] DELETE /identity/v3/projects/86b3133477f941cc8c46c2ad8b09cc8b => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:23.518049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:23.529556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:23.541269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee36d9024964e9c89239970cd5edda9', 'request_id': 'req-e84248a9-3ad5-4772-9310-d79fab467cd1'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.564299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee36d9024964e9c89239970cd5edda9', 'request_id': 'req-e84248a9-3ad5-4772-9310-d79fab467cd1'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.593776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee36d9024964e9c89239970cd5edda9', 'request_id': 'req-e84248a9-3ad5-4772-9310-d79fab467cd1'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.637927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] The token cache is being invalidate because project 3ee36d9024964e9c89239970cd5edda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:23.645031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.645847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:23.650900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 876/1738] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:23 2022] DELETE /identity/v3/projects/3ee36d9024964e9c89239970cd5edda9 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:23.655647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:23.659799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:23.668729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:23.736641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a5e87-ca5a-4092-bae0-f503b1618171 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:23.804652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:23.840750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84248a9-3ad5-4772-9310-d79fab467cd1 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:23.851678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:23.857834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:23.858553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:23.859031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] PATH_INFO: `/v3/users/26936c2772df4930b84c6b886be78000` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:23.860900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:23.861258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26936c2772df4930b84c6b886be78000)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:23.861859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:23.862188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:23.862468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] PATH_INFO: `/v3/users/68d2caca05684ad2abdb274b5f0be305` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:23.865488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d2caca05684ad2abdb274b5f0be305)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:23.866599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:23.869344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:23.979592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26936c2772df4930b84c6b886be78000'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.011051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26936c2772df4930b84c6b886be78000'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.019359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d2caca05684ad2abdb274b5f0be305'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.028848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26936c2772df4930b84c6b886be78000'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.043035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26936c2772df4930b84c6b886be78000'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.045094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26936c2772df4930b84c6b886be78000'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.059002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d2caca05684ad2abdb274b5f0be305'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.063103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 877/1739] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:23 2022] DELETE /identity/v3/users/26936c2772df4930b84c6b886be78000 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:24.067344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:24.071786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:24.094505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d2caca05684ad2abdb274b5f0be305'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.108907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d2caca05684ad2abdb274b5f0be305'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.109684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d2caca05684ad2abdb274b5f0be305'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.129285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 863/1740] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:23 2022] DELETE /identity/v3/users/68d2caca05684ad2abdb274b5f0be305 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:24.130429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:24.136553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:24.246956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619072f7-a6c6-4534-8563-b4eec343af96 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:24.371250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b42b-eab3-43a6-9d7f-15190e776a16 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:24.400493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:24.401961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.403738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.404172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] PATH_INFO: `/v3/users/ed2e0af1dd204bc9a9d8447766f3131d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:24.407720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:24.408844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.409288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.409698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] PATH_INFO: `/v3/users/8247e3b9b8a54a919bf1550ad39b1646` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:24.411794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2e0af1dd204bc9a9d8447766f3131d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:24.412581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8247e3b9b8a54a919bf1550ad39b1646)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:24.417493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:24.418592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:24.468287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:24.487109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:24.569148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2e0af1dd204bc9a9d8447766f3131d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.575880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8247e3b9b8a54a919bf1550ad39b1646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.606972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2e0af1dd204bc9a9d8447766f3131d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.618298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8247e3b9b8a54a919bf1550ad39b1646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.634003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2e0af1dd204bc9a9d8447766f3131d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.648778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8247e3b9b8a54a919bf1550ad39b1646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.661739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2e0af1dd204bc9a9d8447766f3131d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.663481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8247e3b9b8a54a919bf1550ad39b1646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.663938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-aa5f9b84-a061-4591-8702-d8557ba027d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2e0af1dd204bc9a9d8447766f3131d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.666190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ca46a261-b0be-4544-a745-d2380029f059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8247e3b9b8a54a919bf1550ad39b1646'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:24.682561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 878/1741] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:24 2022] DELETE /identity/v3/users/ed2e0af1dd204bc9a9d8447766f3131d => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:24.685598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 864/1742] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:24 2022] DELETE /identity/v3/users/8247e3b9b8a54a919bf1550ad39b1646 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:24.687837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46d13c-10fb-4cd7-8d93-fdb27df80788 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.688162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46d13c-10fb-4cd7-8d93-fdb27df80788 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.688528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46d13c-10fb-4cd7-8d93-fdb27df80788 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:24.689780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d3d1-e10b-46a9-8b9d-95b12a10c91f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.690122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d3d1-e10b-46a9-8b9d-95b12a10c91f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.690381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d3d1-e10b-46a9-8b9d-95b12a10c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:24.692831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 879/1743] 10.222.0.64 () {54 vars in 912 bytes} [Fri Jun 24 10:16:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:24.693924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e53e5-aaa8-4bd1-bcbe-850aad7898c4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.694260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e53e5-aaa8-4bd1-bcbe-850aad7898c4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.694534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e53e5-aaa8-4bd1-bcbe-850aad7898c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:24.759570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5740d3d1-e10b-46a9-8b9d-95b12a10c91f None None] MFA Rules not processed for user `fe9a62b8fcdc400b9ceb993bb8f936e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:24.768316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5740d3d1-e10b-46a9-8b9d-95b12a10c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:24.773256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-333e53e5-aaa8-4bd1-bcbe-850aad7898c4 None None] MFA Rules not processed for user `acbafaab99f848828c304ca5a45a55dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:24.774636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-333e53e5-aaa8-4bd1-bcbe-850aad7898c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:24.966475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 865/1744] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:24.969236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.969706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.970191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:24.987737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 880/1745] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:24.988978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff67128-f0cf-4c34-ad5d-ef771e4a6042 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:24.989268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff67128-f0cf-4c34-ad5d-ef771e4a6042 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:24.989547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff67128-f0cf-4c34-ad5d-ef771e4a6042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.024541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-cff67128-f0cf-4c34-ad5d-ef771e4a6042 None None] MFA Rules not processed for user `fe9a62b8fcdc400b9ceb993bb8f936e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:25.026910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cff67128-f0cf-4c34-ad5d-ef771e4a6042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.034731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 881/1746] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:25.036283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:25.036579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:25.036885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.080912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] MFA Rules not processed for user `acbafaab99f848828c304ca5a45a55dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:25.087010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.099123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:25.102760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.107885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 882/1747] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:25.110159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:25.115178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.244119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 866/1748] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:16:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:25.289104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:25.290769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.307094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d9ecf-35a4-4121-9639-7293d910508e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:25.314085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b855f0-2b9d-4d34-97e8-af058392cdd3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:25.322569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:25.323871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:25.324166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:25.324902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:25.325587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.326444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:25.326762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:25.328318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.346206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:25.355616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:25.356657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:25.358103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:25.360179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:25.380965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 867/1749] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:25.395100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 883/1750] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:25.472026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:25.473193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:25.486118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.487033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.528336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:25.650734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ccd84-b48a-4c24-b58c-877158dc7517 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:25.657185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774af2-b664-4ba6-bc39-7ce568f95134 service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:25.663524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:25.664827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:25.666050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:25.666645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:25.667354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] PATH_INFO: `/v3/projects/73c43ace79c842da946a40e236d5a151` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.668596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:25.669718 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c43ace79c842da946a40e236d5a151)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:25.669894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:25.670344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] PATH_INFO: `/v3/projects/709010bf0d694d9fa40b122230322fe3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.672749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:25.674051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=709010bf0d694d9fa40b122230322fe3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:25.682013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:25.808186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c43ace79c842da946a40e236d5a151', 'request_id': 'req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.827204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c43ace79c842da946a40e236d5a151', 'request_id': 'req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.853780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c43ace79c842da946a40e236d5a151', 'request_id': 'req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.871367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709010bf0d694d9fa40b122230322fe3', 'request_id': 'req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.874988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] The token cache is being invalidate because project 73c43ace79c842da946a40e236d5a151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:25.875861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.878174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.886442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 884/1751] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:25 2022] DELETE /identity/v3/projects/73c43ace79c842da946a40e236d5a151 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:25.889667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:25.893176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:25.900409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709010bf0d694d9fa40b122230322fe3', 'request_id': 'req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.923080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709010bf0d694d9fa40b122230322fe3', 'request_id': 'req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.953555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] The token cache is being invalidate because project 709010bf0d694d9fa40b122230322fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:25.954081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.955810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f09f36c9-a4ad-4448-8cfb-7a452f0c7050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:25.957685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 868/1752] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:25 2022] DELETE /identity/v3/projects/709010bf0d694d9fa40b122230322fe3 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:25.960147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e57ee3-f337-48b6-a7bc-f6b11b1cb3c7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:25.963974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e57ee3-f337-48b6-a7bc-f6b11b1cb3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:25.965265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e57ee3-f337-48b6-a7bc-f6b11b1cb3c7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:25.971781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 869/1753] 10.222.0.64 () {54 vars in 912 bytes} [Fri Jun 24 10:16:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:25.973951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e57ee3-f337-48b6-a7bc-f6b11b1cb3c7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:25.982345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8e57ee3-f337-48b6-a7bc-f6b11b1cb3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.074383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2ccbe-f8ca-46e8-a73b-0678bbbf24ca admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:26.149410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:26.158934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e57ee3-f337-48b6-a7bc-f6b11b1cb3c7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:26.167820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:26.168711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.168994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.169270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] PATH_INFO: `/v3/projects/c334aeb0d6fe499b9caf09111c6f81f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.171033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c334aeb0d6fe499b9caf09111c6f81f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:26.173629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:26.219543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:26.221223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.221511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.222449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] PATH_INFO: `/v3/projects/f8fe2ac64e77463f868efb0de979edcf` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.226811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fe2ac64e77463f868efb0de979edcf)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:26.229828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:26.259063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:26.279126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c334aeb0d6fe499b9caf09111c6f81f5', 'request_id': 'req-2d374d0f-cff3-4c14-b169-aaa96f76dc35'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.306653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c334aeb0d6fe499b9caf09111c6f81f5', 'request_id': 'req-2d374d0f-cff3-4c14-b169-aaa96f76dc35'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.319010 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c334aeb0d6fe499b9caf09111c6f81f5', 'request_id': 'req-2d374d0f-cff3-4c14-b169-aaa96f76dc35'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.350392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] The token cache is being invalidate because project c334aeb0d6fe499b9caf09111c6f81f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:26.351247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.352944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2d374d0f-cff3-4c14-b169-aaa96f76dc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.361606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 870/1754] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:25 2022] DELETE /identity/v3/projects/c334aeb0d6fe499b9caf09111c6f81f5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:26.363331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd0d44-fb80-4096-832e-1d0379a3c889 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.365351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd0d44-fb80-4096-832e-1d0379a3c889 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.365351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd0d44-fb80-4096-832e-1d0379a3c889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.393602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fe2ac64e77463f868efb0de979edcf', 'request_id': 'req-710b1ede-a98d-4f28-a483-b8090a96f14a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.396540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f4fd0d44-fb80-4096-832e-1d0379a3c889 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:26.398298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f4fd0d44-fb80-4096-832e-1d0379a3c889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.407403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fe2ac64e77463f868efb0de979edcf', 'request_id': 'req-710b1ede-a98d-4f28-a483-b8090a96f14a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.430471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fe2ac64e77463f868efb0de979edcf', 'request_id': 'req-710b1ede-a98d-4f28-a483-b8090a96f14a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.447778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] The token cache is being invalidate because project f8fe2ac64e77463f868efb0de979edcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:26.448477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.449659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-710b1ede-a98d-4f28-a483-b8090a96f14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:26.455026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 885/1755] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:25 2022] DELETE /identity/v3/projects/f8fe2ac64e77463f868efb0de979edcf => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:26.458033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35691186-2e37-41d6-a29f-861acc6fb4f5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.458690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35691186-2e37-41d6-a29f-861acc6fb4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.459288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35691186-2e37-41d6-a29f-861acc6fb4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.494112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-35691186-2e37-41d6-a29f-861acc6fb4f5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:26.495920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-35691186-2e37-41d6-a29f-861acc6fb4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.498890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 871/1756] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:26.500998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.502170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.503317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.515569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 886/1757] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:26.516938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20c734-3c20-4af0-87b2-fb8e937ebaad None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.517222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20c734-3c20-4af0-87b2-fb8e937ebaad None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.517505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20c734-3c20-4af0-87b2-fb8e937ebaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.542308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:26.546242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.556974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0c20c734-3c20-4af0-87b2-fb8e937ebaad None None] MFA Rules not processed for user `6ca14b36fc754fe9b24abdfecf0f64c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:26.558244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0c20c734-3c20-4af0-87b2-fb8e937ebaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.676262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 872/1758] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:26.677778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:26.679734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.709159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 887/1759] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:26.717247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454fab7-f7fe-4a14-8720-316b1f9ab900 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:26.717532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.717992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.718421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.725605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:26.726391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.726675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.726994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.734064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.749378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:26.750962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.764864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 888/1760] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:26.766020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:26.777356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.871915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:26.883392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:26.886523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:26.900152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:26.907266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f58ec-1d7c-4166-bf1b-6145aacbe472 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:26.915276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:26.917520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.917520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.917751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.920310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.927363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 873/1761] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:26.932854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:26.939496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:26.942870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:26.944845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012a44a-4bfe-4eec-94da-432e3261bb16 service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:26.969022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 889/1762] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:26.971129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e2fb-6c0f-47bd-a43b-b6c45ec8a105 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.971129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e2fb-6c0f-47bd-a43b-b6c45ec8a105 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.971547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17160642-ee3b-4c0e-ae5b-f3499cacd258 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:26.974221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e2fb-6c0f-47bd-a43b-b6c45ec8a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.977889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:26.979276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:26.981075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:26.983307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] PATH_INFO: `/v3/users/7879ac262ccd439b96fdaa6072412b15` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:26.986164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7879ac262ccd439b96fdaa6072412b15)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:26.993156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:27.021398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-dcc9e2fb-6c0f-47bd-a43b-b6c45ec8a105 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:27.035431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dcc9e2fb-6c0f-47bd-a43b-b6c45ec8a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.072130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 890/1763] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:27.072130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.072130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.072130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.108268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:27.110059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.129241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 891/1764] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:27.130150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:27.131855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.186927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7879ac262ccd439b96fdaa6072412b15'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:27.229827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7879ac262ccd439b96fdaa6072412b15'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:27.246721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7879ac262ccd439b96fdaa6072412b15'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:27.270284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7879ac262ccd439b96fdaa6072412b15'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:27.271140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-20a49446-0e0a-4f6d-9bcb-8b7be72ed988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7879ac262ccd439b96fdaa6072412b15'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:27.300017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 874/1765] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:26 2022] DELETE /identity/v3/users/7879ac262ccd439b96fdaa6072412b15 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:27.322504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.322945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.324136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.345242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38281462-13f8-4532-ba4d-167e1fe77ffd None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:27.406629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:27.407719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.522934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:27.525525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.526132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.526912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.532481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.680383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 875/1766] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:27.683784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:27.687178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.734340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d212-782c-494f-a8f1-52ccae60eafe None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:27.747302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:27.747995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.748272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.748543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.751543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:27.751854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:27.753542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:27.768246 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:27.780404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:27.786630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 876/1767] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:27.786630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:27.796353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:27.809353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 892/1768] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:27.810778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:27.831859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e4050-adc3-4fdd-8f00-68ba520d1cdf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:27.850587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:27.851316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.851595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.851866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.853138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:27.853449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:27.858360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:27.860136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0541930-10f2-4b64-9367-629705bb4cca service glance] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:27.901196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 877/1769] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:27.901438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:27.902092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.902212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:27.902382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.902736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.903450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:27.903744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:27.916184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:27.937303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88fe0e-267e-40d6-9c8d-9e67bf262132 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:27.961917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:27.962908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:27.963362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:27.963774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:27.964641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:27.967241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:27.980488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 893/1770] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:27.981479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:28.019859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888a091-d5ae-4765-b955-eb69f3ad0c5e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:28.040795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 878/1771] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:27 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:28.040795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:28.053007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:28.054384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:28.054384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:28.054384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:28.056675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:28.059061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:28.123342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:28.144205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:28.205125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 894/1772] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:27 2022] POST /identity/v3/projects => generated 409 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:28.206158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:28.309417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fece3-db76-4cdf-a220-751d8c1be178 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:28.451599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4675b-3a4c-4d88-b00b-63816172e5c0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:28.467883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:28.468889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:28.469241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:28.469592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:28.470469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:28.473541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:28.505129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:28.506171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:28.506558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:28.506881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:28.507612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:28.515475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:28.521746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 895/1773] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:28 2022] POST /identity/v3/users => generated 287 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:28.522664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:28.528918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:28.562578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b548d2-ba23-4a02-ae6c-b708e818d59a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:28.576732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:28.579124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:28.580122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:28.586199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 879/1774] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:28 2022] POST /identity/v3/projects => generated 381 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:28.586471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] PATH_INFO: `/v3/projects/fb85430ae70247f084760e63fc517114` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:28.587145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:28.589511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb85430ae70247f084760e63fc517114)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:28.598466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:28.819298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e2c86-2422-41af-bba0-dd1d12f31d33 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:28.833597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:28.834351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:28.834649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:28.837553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:28.838303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:28.842672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:28.915353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 880/1775] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:28 2022] POST /identity/v3/users => generated 305 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:28.915353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:28.965454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447dedd-56ac-4aed-a6bc-ad4466e940a6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:28.997047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:28.997753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:28.998037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:28.998310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:28.999133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.001942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.015399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 881/1776] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:29.016567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.041269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce64f8b-3c11-4527-a9ae-ae944dc72cd8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.055703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e786-4a14-4dd9-84ed-a7e909541234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.056792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e786-4a14-4dd9-84ed-a7e909541234 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.057255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e786-4a14-4dd9-84ed-a7e909541234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.057694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e786-4a14-4dd9-84ed-a7e909541234 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.058559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4e786-4a14-4dd9-84ed-a7e909541234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.062359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4e786-4a14-4dd9-84ed-a7e909541234 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.116749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb85430ae70247f084760e63fc517114', 'request_id': 'req-382b967a-6515-45ba-bd68-af1163629b2b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:29.128610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 882/1777] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:29 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:29.130504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48177750-8fb4-400e-92c9-457485bddcd8 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.130824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48177750-8fb4-400e-92c9-457485bddcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.131105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48177750-8fb4-400e-92c9-457485bddcd8 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.134083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb85430ae70247f084760e63fc517114', 'request_id': 'req-382b967a-6515-45ba-bd68-af1163629b2b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:29.136793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 883/1778] 10.222.0.64 () {54 vars in 912 bytes} [Fri Jun 24 10:16:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:29.138198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48177750-8fb4-400e-92c9-457485bddcd8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.160742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb85430ae70247f084760e63fc517114', 'request_id': 'req-382b967a-6515-45ba-bd68-af1163629b2b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:29.181265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:29.188372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] The token cache is being invalidate because project fb85430ae70247f084760e63fc517114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:29.188692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:29.192315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:29.195908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 896/1779] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:28 2022] DELETE /identity/v3/projects/fb85430ae70247f084760e63fc517114 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:29.198309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.201234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.268408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48177750-8fb4-400e-92c9-457485bddcd8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.308416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.308416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.308416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.308416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.308874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.308874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.333012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 884/1780] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:29.334112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.336225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.353421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c1d59-39d9-4e8a-bd67-0d670fbad17f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.357836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b967a-6515-45ba-bd68-af1163629b2b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.359760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38722b-91e6-4803-8295-989426b08107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.360816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38722b-91e6-4803-8295-989426b08107 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.361310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38722b-91e6-4803-8295-989426b08107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.361748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38722b-91e6-4803-8295-989426b08107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.362954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bc595-5b47-4b8c-8c6b-802836db74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.363363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38722b-91e6-4803-8295-989426b08107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.365339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38722b-91e6-4803-8295-989426b08107 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.365774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc595-5b47-4b8c-8c6b-802836db74e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.366230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc595-5b47-4b8c-8c6b-802836db74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.366569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc595-5b47-4b8c-8c6b-802836db74e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.367506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bc595-5b47-4b8c-8c6b-802836db74e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.369654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bc595-5b47-4b8c-8c6b-802836db74e8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.378844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:29.381081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 885/1781] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:29.382571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.384114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.384454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.394533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 897/1782] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:29.397590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.398540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.399376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.413165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:29.418382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.454731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:29.458895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.487260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 898/1783] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:29.487925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.492621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.543622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7411e1d-1044-476c-8565-63473a193e11 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.571854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.573366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.577761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.578220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.580305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.582092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.583365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 886/1784] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:29.584376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.613281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68157f-d9e8-4288-bfd6-aa737de244c9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.629737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 899/1785] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:29.631119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5d7b-7539-40c6-9374-ff567545a341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.632263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5d7b-7539-40c6-9374-ff567545a341 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.632939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5d7b-7539-40c6-9374-ff567545a341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.633516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5d7b-7539-40c6-9374-ff567545a341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.635302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5d7b-7539-40c6-9374-ff567545a341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.636017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.637398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5d7b-7539-40c6-9374-ff567545a341 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.667748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 887/1786] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:29.669485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.669957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.670418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.679349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed10c2-92c0-4f17-8f6d-9cc88291db1b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.698596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:29.699970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.706974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.706974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.706974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.706974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] PATH_INFO: `/v3/projects/a8c521bf29f14b9db28816bed9a4d0f3/users/2934818551254f8093086dd94eef5e08/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.710196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 888/1787] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:29.711147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.714677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:29.731920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979ce11-b709-49b0-9107-f133269b69ff None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.738113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.738894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.739460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.739832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.753787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c521bf29f14b9db28816bed9a4d0f3, user_id=2934818551254f8093086dd94eef5e08, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.756327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:29.757222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.758673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:29.779060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 889/1788] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:29.784835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.808875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750a46d-ad28-4e12-8f10-bd12b8f10080 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:29.817209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 900/1789] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:29 2022] PUT /identity/v3/projects/a8c521bf29f14b9db28816bed9a4d0f3/users/2934818551254f8093086dd94eef5e08/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:29.818213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:29.979415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:29.980508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:29.980796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:29.981087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] PATH_INFO: `/v3/projects/29dd71f687a24c4eb04278fda56e523f/users/0bc8e210bef54261a434c12a37bb1c2f/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:29.987366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b6759-e3cd-47ed-9fae-84f0655f7c76 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:30.004026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342799bc-fa5f-4d88-8dba-4203c81e36e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:30.004967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342799bc-fa5f-4d88-8dba-4203c81e36e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.005249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342799bc-fa5f-4d88-8dba-4203c81e36e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.005525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342799bc-fa5f-4d88-8dba-4203c81e36e1 admin admin] PATH_INFO: `/v3/projects/f20216fd843d445a862f4eacfe34692c/users/38d7f9a49dd249869e5c92b617345415/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.014985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] RBAC: Authorizing `identity:create_grant(project_id=29dd71f687a24c4eb04278fda56e523f, user_id=0bc8e210bef54261a434c12a37bb1c2f, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:30.023306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:30.045703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342799bc-fa5f-4d88-8dba-4203c81e36e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20216fd843d445a862f4eacfe34692c, user_id=38d7f9a49dd249869e5c92b617345415, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:30.060796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342799bc-fa5f-4d88-8dba-4203c81e36e1 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:30.071425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 890/1790] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:29 2022] PUT /identity/v3/projects/29dd71f687a24c4eb04278fda56e523f/users/0bc8e210bef54261a434c12a37bb1c2f/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:30.071425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:30.083108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:30.105971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 901/1791] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:29 2022] PUT /identity/v3/projects/f20216fd843d445a862f4eacfe34692c/users/38d7f9a49dd249869e5c92b617345415/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:30.109526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171be349-387f-4d6e-9620-8a7479685cca None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.110910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171be349-387f-4d6e-9620-8a7479685cca None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.111292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171be349-387f-4d6e-9620-8a7479685cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.171615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-171be349-387f-4d6e-9620-8a7479685cca None None] MFA Rules not processed for user `2934818551254f8093086dd94eef5e08`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:30.173793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-171be349-387f-4d6e-9620-8a7479685cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:30.237949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19875830-3b35-407c-a8e7-345b6878952f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:30.359138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:30.363722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.364261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.364385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 902/1792] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:30.372716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.373603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:30.373804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.373904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:30.374398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.374902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.375605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cc113-0219-4a69-b96e-0224af3f8388 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:30.393828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 891/1793] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:30.395514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.395970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.396250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.468563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] MFA Rules not processed for user `38d7f9a49dd249869e5c92b617345415`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:30.472443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:30.505544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] MFA Rules not processed for user `0bc8e210bef54261a434c12a37bb1c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:30.510319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:30.569951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:30.683846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 892/1794] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:30.685692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:30.707180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130512-3633-49ad-96fb-748cac44b3f8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:30.725020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:30.725676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.725957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.726227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.727012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:30.729516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:30.740116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 903/1795] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:30.742228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:30.820210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef860a-f3c6-481d-ac3a-9e4d1e158941 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:30.838932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:30.841392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:30.842031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:30.842303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:30.847745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:30.849828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:30.858312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 893/1796] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:30.860189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:30.897044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:30.927491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 904/1797] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:30.930565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:31.090771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7853f1-49c3-4782-b44d-432390ea59e2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:31.158944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f6a57-b7bb-464b-af2c-4ba596c7bf1a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:31.178104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:31.180009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.185344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.185887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.186856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:31.189139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:31.236311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:31.236541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.236791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.237060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.237767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:31.241465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:31.286205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 905/1798] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/projects => generated 407 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:31.287601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:31.298823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 894/1799] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:30 2022] POST /identity/v3/projects => generated 381 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:31.303656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:31.445130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f426a9-37c0-48c2-bd71-e045e167b76b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:31.453178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:31.454000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.478953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.480454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.485026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:31.525575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9348e5-a2bb-4c7a-a02f-252d31538593 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:31.541534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7e4d6-a812-4c33-aa3c-a458d463b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:31.542245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7e4d6-a812-4c33-aa3c-a458d463b63a admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.542573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7e4d6-a812-4c33-aa3c-a458d463b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.542972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7e4d6-a812-4c33-aa3c-a458d463b63a admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.543694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7e4d6-a812-4c33-aa3c-a458d463b63a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:31.545714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7e4d6-a812-4c33-aa3c-a458d463b63a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:31.546007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:31.599826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 906/1800] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:31 2022] POST /identity/v3/users => generated 292 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:31.603316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33f2d6-0f89-43dc-bbf6-fe72a82e9070 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.603821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33f2d6-0f89-43dc-bbf6-fe72a82e9070 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.604753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33f2d6-0f89-43dc-bbf6-fe72a82e9070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.633019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-da33f2d6-0f89-43dc-bbf6-fe72a82e9070 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:31.635257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-da33f2d6-0f89-43dc-bbf6-fe72a82e9070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:31.650966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 895/1801] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:31 2022] POST /identity/v3/users => generated 294 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:31.652089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:31.702179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394606da-822f-48c2-bb4e-a1d991576ae6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:31.714370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:31.732699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:31.733738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.734045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.736798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.737790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:31.742534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:31.847261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 896/1802] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:31 2022] POST /identity/v3/users => generated 310 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:31.848172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:31.885542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb932f0-1dc0-4782-a994-7141fd51aac1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:31.890548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 907/1803] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:31.892154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.892452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.892887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.922650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:31.923905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:31.925520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:31.926085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:31.927781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:31.956198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:31.975744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] MFA Rules not processed for user `acbafaab99f848828c304ca5a45a55dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:31.979497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:32.021721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:32.041541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 897/1804] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:31 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:32.042668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.071646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd184a9-2192-439a-a0cc-def6f6cb0293 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.094525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.100208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.100797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.101520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.105830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.111070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.148135 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 898/1805] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:32.151603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.164031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 908/1806] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:32.167206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.187754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef91fab-d184-47f0-a32a-8e53c091205e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.200284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6e435-7781-40b6-ba63-49c61d9aa006 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.213470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.214467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.214969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.215464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.217105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.220235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.225131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.226009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.226296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.226692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.228585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.236346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.257600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 909/1807] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:32.259275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.276091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 899/1808] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:32.276091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.305904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281305aa-19bf-437c-bc5c-ffd85aa86f15 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.309471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03cfe5-d262-4eaf-b989-897383d7aadb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.331985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.333002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.333390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.333784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.342175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.343059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.344879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.346497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.346497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.348009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] PATH_INFO: `/v3/projects/7415a5010971428a9582b84bf6af8376/users/8117614a253346bda80dee254726fdd6/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.368734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 910/1809] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:32.370061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.374028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:32.403745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7415a5010971428a9582b84bf6af8376, user_id=8117614a253346bda80dee254726fdd6, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.412731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.461470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 900/1810] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:32 2022] PUT /identity/v3/projects/7415a5010971428a9582b84bf6af8376/users/8117614a253346bda80dee254726fdd6/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:32.462339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.610472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e865f20-5411-4350-b8ec-4f6193a383e9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.657390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fcca4-d818-4179-a8a8-4a131945375f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.672696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.673626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.674033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.674311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] PATH_INFO: `/v3/projects/0947020c6daa42308f6ec5da4001ac77/users/3e23090c3c6f4be9ad3ff1d5379a6d55/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.737689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:32.740938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0947020c6daa42308f6ec5da4001ac77, user_id=3e23090c3c6f4be9ad3ff1d5379a6d55, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.743718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.783074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 901/1811] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:32 2022] PUT /identity/v3/projects/0947020c6daa42308f6ec5da4001ac77/users/3e23090c3c6f4be9ad3ff1d5379a6d55/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:32.785592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:32.796267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.796958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.797239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.797981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.937660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.940620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.947190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921c8f3-2a84-4674-b64c-c04ccd586512 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:32.956154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:32.956856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:32.957129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:32.957406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:32.958155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:32.959842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:32.981411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 911/1812] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:16:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:16:32.982778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.043130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c9ad8-9114-4d22-9aa9-419ed409923d service swift] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.044776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 902/1813] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:33.046066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.058674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:33.061149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:33.061440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:33.061779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] PATH_INFO: `/v3/projects/2a7ccd504f3e48d3aee8dd35e50a989f/users/f50cfa71c0184531b13f0fa1ef85d00b/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:33.074467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f2c5c-d00d-4b5f-8701-8065e37e0460 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.095744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:33.098855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:33.099176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:33.099455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:33.100626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:33.106577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:33.114075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7ccd504f3e48d3aee8dd35e50a989f, user_id=f50cfa71c0184531b13f0fa1ef85d00b, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:33.121455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:33.133095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 903/1814] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:33.134627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.161624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 912/1815] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:32 2022] PUT /identity/v3/projects/2a7ccd504f3e48d3aee8dd35e50a989f/users/f50cfa71c0184531b13f0fa1ef85d00b/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:33.163457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.171497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b86dd3-a1b8-4f9d-8e35-6b6f8987000a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.264489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:33.446859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:33.447935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:33.448385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:33.448808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:33.450456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:33.452756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:33.469897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 904/1816] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:33.471143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.486538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7053f-13da-4946-bc3a-2c96b5bfc3c1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.492448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:33.494811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:33.495314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:33.496393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] PATH_INFO: `/v3/projects/7415a5010971428a9582b84bf6af8376/users/8117614a253346bda80dee254726fdd6/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:33.499730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7415a5010971428a9582b84bf6af8376, user_id=8117614a253346bda80dee254726fdd6, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:33.504689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e21077-324c-4600-b503-8caf6efe119e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.505405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:33.537406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 905/1817] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:33 2022] PUT /identity/v3/projects/7415a5010971428a9582b84bf6af8376/users/8117614a253346bda80dee254726fdd6/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:33.538682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.549803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:33.551740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:33.553241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:33.553584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] PATH_INFO: `/v3/projects/b1536354ea36428082a8f549a85ed09e/users/eb43a1609d394319859b50db9d46a5c8/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:33.611016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1536354ea36428082a8f549a85ed09e, user_id=eb43a1609d394319859b50db9d46a5c8, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:33.625001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:33.672007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 913/1818] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:33 2022] PUT /identity/v3/projects/b1536354ea36428082a8f549a85ed09e/users/eb43a1609d394319859b50db9d46a5c8/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:33.673253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:33.823003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eff92f-b0a8-4e1e-956e-671491cf5253 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.854989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:33.927882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f6259-0196-4c85-888b-0c174e17710c admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:33.955171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb65d23-524a-4416-a891-2c1edc831311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:33.956383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65d23-524a-4416-a891-2c1edc831311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:33.957337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65d23-524a-4416-a891-2c1edc831311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:33.957337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65d23-524a-4416-a891-2c1edc831311 admin admin] PATH_INFO: `/v3/projects/0947020c6daa42308f6ec5da4001ac77/users/3e23090c3c6f4be9ad3ff1d5379a6d55/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:33.971606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb65d23-524a-4416-a891-2c1edc831311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0947020c6daa42308f6ec5da4001ac77, user_id=3e23090c3c6f4be9ad3ff1d5379a6d55, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:33.988179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb65d23-524a-4416-a891-2c1edc831311 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:34.025177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 914/1819] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:33 2022] PUT /identity/v3/projects/0947020c6daa42308f6ec5da4001ac77/users/3e23090c3c6f4be9ad3ff1d5379a6d55/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:34.028437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442e1d4-a7a6-4382-a6bc-bb2d6b6d05e4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:34.028729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442e1d4-a7a6-4382-a6bc-bb2d6b6d05e4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:34.029008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442e1d4-a7a6-4382-a6bc-bb2d6b6d05e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:34.041314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e6c12-8595-480b-8d5b-c01596db35d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:34.042189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6c12-8595-480b-8d5b-c01596db35d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:34.042448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6c12-8595-480b-8d5b-c01596db35d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:34.042739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6c12-8595-480b-8d5b-c01596db35d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:34.043497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6c12-8595-480b-8d5b-c01596db35d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:34.045229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6c12-8595-480b-8d5b-c01596db35d0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:34.075877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 906/1820] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 538 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:34.084257 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:34.084688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:34.084958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:34.155595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9442e1d4-a7a6-4382-a6bc-bb2d6b6d05e4 None None] MFA Rules not processed for user `8117614a253346bda80dee254726fdd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:34.162147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9442e1d4-a7a6-4382-a6bc-bb2d6b6d05e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:34.219702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] MFA Rules not processed for user `eb43a1609d394319859b50db9d46a5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:34.222317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:34.340533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:34.389985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 915/1821] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:34.393958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f5714-75c0-4c96-a617-346f14d385da None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:34.394244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f5714-75c0-4c96-a617-346f14d385da None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:34.394761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f5714-75c0-4c96-a617-346f14d385da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:34.421456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 907/1822] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:34.425077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:34.513285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-407f5714-75c0-4c96-a617-346f14d385da None None] MFA Rules not processed for user `3e23090c3c6f4be9ad3ff1d5379a6d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:34.519054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-407f5714-75c0-4c96-a617-346f14d385da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:34.704692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a90758-5f6a-4e09-9755-d3e27ed5c90a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:34.723711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:34.724564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:34.724868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:34.725113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] PATH_INFO: `/v3/projects/2a7ccd504f3e48d3aee8dd35e50a989f/users/f50cfa71c0184531b13f0fa1ef85d00b/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:34.734604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7ccd504f3e48d3aee8dd35e50a989f, user_id=f50cfa71c0184531b13f0fa1ef85d00b, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:34.741775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:34.788564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 916/1823] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:34 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:34.788564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f5714-75c0-4c96-a617-346f14d385da None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:34.798544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 908/1824] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:34 2022] PUT /identity/v3/projects/2a7ccd504f3e48d3aee8dd35e50a989f/users/f50cfa71c0184531b13f0fa1ef85d00b/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:34.799981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:34.805868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-407f5714-75c0-4c96-a617-346f14d385da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:34.961117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:34.968133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f5714-75c0-4c96-a617-346f14d385da None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:35.080664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:35.082154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.082669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.083216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.117729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64c6a9-db8f-4b57-aff7-e23435bb62bb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:35.153183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c9610-f211-407c-aa90-ae0c35346acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:35.153885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c9610-f211-407c-aa90-ae0c35346acf admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.154218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c9610-f211-407c-aa90-ae0c35346acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.157095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c9610-f211-407c-aa90-ae0c35346acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.158906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c9610-f211-407c-aa90-ae0c35346acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:35.161219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c9610-f211-407c-aa90-ae0c35346acf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:35.183102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:35.185890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:35.217420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 917/1825] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:35.220812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:35.224204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:35.239438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 909/1826] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:34 2022] POST /identity/v3/projects => generated 387 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:35.240723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5587e2-7b7f-40b5-b7a9-e7d87c931cc9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.241088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5587e2-7b7f-40b5-b7a9-e7d87c931cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.241360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5587e2-7b7f-40b5-b7a9-e7d87c931cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.249583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446f3e6-d408-4b4b-9fd9-e8903065c206 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:35.351618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0e5587e2-7b7f-40b5-b7a9-e7d87c931cc9 None None] MFA Rules not processed for user `f50cfa71c0184531b13f0fa1ef85d00b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:35.353642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0e5587e2-7b7f-40b5-b7a9-e7d87c931cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:35.450781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:35.451484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.452117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.452419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.506324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:35.575518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 910/1827] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:35.579553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.584399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.584755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.647252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:35.649405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:35.699257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:35.704961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:35.771516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 911/1828] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Jun 24 10:16:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:35.772857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:35.775581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 918/1829] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:35 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:35.777245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:35.810372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:35.823982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300430-efcb-476e-adce-c440d6cdd96e service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:35.832436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:35.833157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.833444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.835256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.951969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439fd04-6593-4416-b17b-ac650b489b43 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:35.966359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:35.969434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:35.969717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:35.969989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:35.974151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:35.989870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.034203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.038099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.040128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 912/1830] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:35 2022] POST /identity/v3/users => generated 300 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:36.041307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.044541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:36.104628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c468b4e-e507-42df-bf77-168f53836d94 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.111976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 919/1831] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:36.116025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.125966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.126677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.127001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.127299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.161040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737b6a-1124-4347-a430-3dfd4453ca97 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.163179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.165013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.179007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.179733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.180065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.180356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.192942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.194586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.203481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 913/1832] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:36.204500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.210502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 920/1833] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:36.211792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.244968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd5c6-6cad-43eb-b5d1-160694509f33 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.246290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f8ec-5024-4800-aa3c-ea84ffbcc3f1 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.261004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.263434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.264206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.265325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.268014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.268669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.268799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.268988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.269969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.274773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.301150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.304786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 921/1834] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:36.304972 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.306131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.329179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 914/1835] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:36.348833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b42206-f944-473d-b4db-823ae626dcb5 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.379316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.380604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.381080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.381845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] PATH_INFO: `/v3/projects/b0696bf798864d07890228b54b5b904b/users/dd9db0abc407407bb0ae5eff9f5bbb9c/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.471803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0696bf798864d07890228b54b5b904b, user_id=dd9db0abc407407bb0ae5eff9f5bbb9c, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.482470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.527445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.527671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 922/1836] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:36 2022] PUT /identity/v3/projects/b0696bf798864d07890228b54b5b904b/users/dd9db0abc407407bb0ae5eff9f5bbb9c/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:36.543319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926343e-5964-4587-89cc-004a70e776b4 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.699029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:36.704675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.720329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.721028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.723558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.724840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.728056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:36.735626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:36.755624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 915/1837] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:16:36.756780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:36.851595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c1323-c750-40bc-a4ba-2ab15a0c6c4e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.863598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.864393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.864748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.865038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.901777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b769c-5ad2-43d7-a773-ea255b4909fc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:36.910198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba1909-9d71-4636-a1c7-f3f1e298c94a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:36.911559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba1909-9d71-4636-a1c7-f3f1e298c94a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:36.912068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba1909-9d71-4636-a1c7-f3f1e298c94a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:36.912687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba1909-9d71-4636-a1c7-f3f1e298c94a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:36.999438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:37.001179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:37.025080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 923/1838] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:37.026742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:37.061508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c2331-ebc1-4e6c-a908-182814926b72 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:37.069801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:37.071457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:37.071859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:37.072319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] PATH_INFO: `/v3/projects/b0696bf798864d07890228b54b5b904b/users/dd9db0abc407407bb0ae5eff9f5bbb9c/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:37.074656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0696bf798864d07890228b54b5b904b, user_id=dd9db0abc407407bb0ae5eff9f5bbb9c, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:37.080450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:37.083199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba1909-9d71-4636-a1c7-f3f1e298c94a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:37.085023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba1909-9d71-4636-a1c7-f3f1e298c94a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:37.091329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:37.099485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 916/1839] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:37.120056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 924/1840] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:37 2022] PUT /identity/v3/projects/b0696bf798864d07890228b54b5b904b/users/dd9db0abc407407bb0ae5eff9f5bbb9c/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:37.126984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:37.127694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:37.128177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:37.240294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] MFA Rules not processed for user `dd9db0abc407407bb0ae5eff9f5bbb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:37.245142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:37.562178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 917/1841] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:37.715282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:37.845198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f558452-218f-4c9c-9a26-883511fc0559 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:37.854280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e86f4-4d16-46b4-9448-6ea5bdc14ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:37.855165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e86f4-4d16-46b4-9448-6ea5bdc14ebb service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:37.855549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e86f4-4d16-46b4-9448-6ea5bdc14ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:37.855940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e86f4-4d16-46b4-9448-6ea5bdc14ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:37.882304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e86f4-4d16-46b4-9448-6ea5bdc14ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:37.884204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e86f4-4d16-46b4-9448-6ea5bdc14ebb service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:37.907951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 925/1842] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:38.333022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:38.419411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:38.524011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cacc3-9811-4f84-a807-191e3848730d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:38.540101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:38.541886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:38.542206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:38.543327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] PATH_INFO: `/v3/users/eb43a1609d394319859b50db9d46a5c8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:38.548628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb43a1609d394319859b50db9d46a5c8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:38.552173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:38.736729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb43a1609d394319859b50db9d46a5c8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:38.795795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb43a1609d394319859b50db9d46a5c8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:38.818242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb43a1609d394319859b50db9d46a5c8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:38.855572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb43a1609d394319859b50db9d46a5c8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:38.858268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb43a1609d394319859b50db9d46a5c8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:38.882836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 918/1843] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:38 2022] DELETE /identity/v3/users/eb43a1609d394319859b50db9d46a5c8 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:38.890183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:38.890936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:38.892444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:38.975464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:38.976649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:39.285656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 926/1844] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:39.408105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:39.409762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:39.665773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a2dc5-4de7-44de-991e-5ce3f96ff075 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:39.699904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2519-f89d-4796-8a97-328545e7bd12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:39.700154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2519-f89d-4796-8a97-328545e7bd12 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:39.700528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2519-f89d-4796-8a97-328545e7bd12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:39.700669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2519-f89d-4796-8a97-328545e7bd12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:39.746205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2519-f89d-4796-8a97-328545e7bd12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:39.747667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2519-f89d-4796-8a97-328545e7bd12 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:39.791924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 919/1845] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:40.831442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:40.840303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:40.870994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898ccf3-f38e-4f22-b5c7-bcb4063dce1b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:40.888280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:40.889772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:40.890525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:40.891169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] PATH_INFO: `/v3/users/38d7f9a49dd249869e5c92b617345415` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:40.894319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d7f9a49dd249869e5c92b617345415)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:40.901633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:41.027308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d7f9a49dd249869e5c92b617345415'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:41.045622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d7f9a49dd249869e5c92b617345415'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:41.058655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d7f9a49dd249869e5c92b617345415'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:41.077978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d7f9a49dd249869e5c92b617345415'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:41.079908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6359c60d-3467-473c-bcc1-c427b9827c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d7f9a49dd249869e5c92b617345415'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:41.107587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 927/1846] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:40 2022] DELETE /identity/v3/users/38d7f9a49dd249869e5c92b617345415 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:41.110639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda78bd4-f90a-4b7b-b373-3fa3fabc0982 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:41.112282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:41.112729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:41.113733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:41.114656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda78bd4-f90a-4b7b-b373-3fa3fabc0982 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:41.115155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda78bd4-f90a-4b7b-b373-3fa3fabc0982 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:41.116551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 920/1847] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:16:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:41.141309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:41.141585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:41.142021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:41.163478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:41.167031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:41.186236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:41.187886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:41.188003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:41.281977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:41.307346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 928/1848] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:41.320032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:41.325618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:41.331753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 921/1849] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:16:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:41.417057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05002a0-1d16-460a-b097-cdb506d58ba6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:41.423930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:41.424682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:41.425084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:41.425213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:41.447230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:41.448770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:41.462672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 929/1850] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:41.796239 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:41.798447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:41.819775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6f74d-527c-485a-8bc8-0732707a9e87 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:41.830204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:41.833299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:41.833791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:41.834255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:41.853380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:41.856994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:41.869602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 922/1851] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:41.885103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:41.888159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:41.915191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ec2b9-9c6f-4cb6-9542-c74996bbde98 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:41.928528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:41.929693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:41.930323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:41.931373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] PATH_INFO: `/v3/projects/f20216fd843d445a862f4eacfe34692c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:41.943657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20216fd843d445a862f4eacfe34692c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:41.946037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:41.978923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:41.980706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:42.020814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e28d3-d6cd-490c-8fb9-72c4f08e0efc service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:42.062678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20216fd843d445a862f4eacfe34692c', 'request_id': 'req-5c59c154-ae0d-4761-94c3-0cea65c30a3a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.075356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20216fd843d445a862f4eacfe34692c', 'request_id': 'req-5c59c154-ae0d-4761-94c3-0cea65c30a3a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.116920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20216fd843d445a862f4eacfe34692c', 'request_id': 'req-5c59c154-ae0d-4761-94c3-0cea65c30a3a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.121178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:42.127593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:42.128622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:42.130414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] PATH_INFO: `/v3/projects/b1536354ea36428082a8f549a85ed09e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:42.136007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1536354ea36428082a8f549a85ed09e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:42.138399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:42.139503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] The token cache is being invalidate because project f20216fd843d445a862f4eacfe34692c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:42.139803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.140541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.142447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 930/1852] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:41 2022] DELETE /identity/v3/projects/f20216fd843d445a862f4eacfe34692c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:42.146242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:42.147965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:42.171128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c154-ae0d-4761-94c3-0cea65c30a3a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:42.177798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:42.180066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:42.180610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:42.181197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] PATH_INFO: `/v3/users/f50cfa71c0184531b13f0fa1ef85d00b` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:42.183032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50cfa71c0184531b13f0fa1ef85d00b)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:42.186404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:42.240502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1536354ea36428082a8f549a85ed09e', 'request_id': 'req-4c14d4f5-ba3a-4f74-a729-64833a81bacf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.247060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1536354ea36428082a8f549a85ed09e', 'request_id': 'req-4c14d4f5-ba3a-4f74-a729-64833a81bacf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.255176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1536354ea36428082a8f549a85ed09e', 'request_id': 'req-4c14d4f5-ba3a-4f74-a729-64833a81bacf'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.275875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] The token cache is being invalidate because project b1536354ea36428082a8f549a85ed09e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:42.276549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.277745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.277922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:42.281454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 923/1853] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:41 2022] DELETE /identity/v3/projects/b1536354ea36428082a8f549a85ed09e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:42.287139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:42.289538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:42.306402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50cfa71c0184531b13f0fa1ef85d00b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.363179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50cfa71c0184531b13f0fa1ef85d00b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.391559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14d4f5-ba3a-4f74-a729-64833a81bacf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:42.392462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50cfa71c0184531b13f0fa1ef85d00b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.397073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:42.398005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:42.398285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:42.398559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] PATH_INFO: `/v3/users/dd9db0abc407407bb0ae5eff9f5bbb9c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:42.399959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9db0abc407407bb0ae5eff9f5bbb9c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:42.401987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:42.405552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:42.417085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50cfa71c0184531b13f0fa1ef85d00b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.419924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6c939962-cfcc-480f-8f85-dcb5e2a57c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50cfa71c0184531b13f0fa1ef85d00b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.437262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 931/1854] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:42 2022] DELETE /identity/v3/users/f50cfa71c0184531b13f0fa1ef85d00b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:42.493486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9db0abc407407bb0ae5eff9f5bbb9c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.512066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9db0abc407407bb0ae5eff9f5bbb9c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.520930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9db0abc407407bb0ae5eff9f5bbb9c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.529696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9db0abc407407bb0ae5eff9f5bbb9c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.530446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b0325590-d78b-4ff2-8aa7-13d1555eedbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9db0abc407407bb0ae5eff9f5bbb9c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:42.537723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 924/1855] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:42 2022] DELETE /identity/v3/users/dd9db0abc407407bb0ae5eff9f5bbb9c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:42.818564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d760b57-9583-4976-8deb-23dfd6cf8720 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:42.818906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d760b57-9583-4976-8deb-23dfd6cf8720 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:42.819188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d760b57-9583-4976-8deb-23dfd6cf8720 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:42.821348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 932/1856] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:16:42.848953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:42.849248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:42.849536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:42.902988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:42.906593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:43.115666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 925/1857] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:16:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:43.142092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d760b57-9583-4976-8deb-23dfd6cf8720 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:43.145034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3d760b57-9583-4976-8deb-23dfd6cf8720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:43.178122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d760b57-9583-4976-8deb-23dfd6cf8720 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:43.194361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95065a0b-42d7-49c9-bd6e-16619204c85e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:43.195149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95065a0b-42d7-49c9-bd6e-16619204c85e service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.195531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95065a0b-42d7-49c9-bd6e-16619204c85e service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.196232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95065a0b-42d7-49c9-bd6e-16619204c85e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.221259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95065a0b-42d7-49c9-bd6e-16619204c85e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:43.223109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95065a0b-42d7-49c9-bd6e-16619204c85e service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:43.261327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:43.263945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:43.290393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 933/1858] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:43.483341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913bbe0-5c8c-4a7e-9114-3edf00f283a6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:43.490348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:43.505484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.505782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.506184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] PATH_INFO: `/v3/projects/2a7ccd504f3e48d3aee8dd35e50a989f` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.507717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7ccd504f3e48d3aee8dd35e50a989f)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:43.510168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:43.532096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:43.618579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7ccd504f3e48d3aee8dd35e50a989f', 'request_id': 'req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.624868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7ccd504f3e48d3aee8dd35e50a989f', 'request_id': 'req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.631863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7ccd504f3e48d3aee8dd35e50a989f', 'request_id': 'req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.647094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] The token cache is being invalidate because project 2a7ccd504f3e48d3aee8dd35e50a989f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:43.647669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.649594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.652182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 926/1859] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:43 2022] DELETE /identity/v3/projects/2a7ccd504f3e48d3aee8dd35e50a989f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:16:43.658036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c347-caff-4955-a32f-d7adb7c7ec54 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.658355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c347-caff-4955-a32f-d7adb7c7ec54 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.658877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350c347-caff-4955-a32f-d7adb7c7ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.686521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5350c347-caff-4955-a32f-d7adb7c7ec54 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:43.690078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5350c347-caff-4955-a32f-d7adb7c7ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:43.710153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:43.711974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:43.799456 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 934/1860] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:43.807472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.808001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.808435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.827548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:43.827778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fddcfb-39be-4a30-bb6b-27d0a6e29bee admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:43.828776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:43.833445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:43.834600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.835014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.835393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] PATH_INFO: `/v3/projects/b0696bf798864d07890228b54b5b904b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.837224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0696bf798864d07890228b54b5b904b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:43.837364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 935/1861] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:43.839509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:43.841933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:43.851765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dbfb-871c-4d88-8601-2b8c55aa6e2b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:43.857412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:43.858168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.858512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.858843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.859578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:43.859886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:43.861680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:43.875532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 936/1862] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:43.879868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:43.891879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de0029-60b4-4211-9aa0-3319e563772b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:43.897328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:43.898056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.898338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.898673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.899636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:43.901754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:43.912040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0696bf798864d07890228b54b5b904b', 'request_id': 'req-ebeed90b-e50c-4eb2-82d0-211a22933aa8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.920592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0696bf798864d07890228b54b5b904b', 'request_id': 'req-ebeed90b-e50c-4eb2-82d0-211a22933aa8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.927824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0696bf798864d07890228b54b5b904b', 'request_id': 'req-ebeed90b-e50c-4eb2-82d0-211a22933aa8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.935341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 937/1863] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:43 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:16:43.940410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:43.943614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] The token cache is being invalidate because project b0696bf798864d07890228b54b5b904b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:43.943907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.944548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ebeed90b-e50c-4eb2-82d0-211a22933aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:43.950522 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 927/1864] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:43 2022] DELETE /identity/v3/projects/b0696bf798864d07890228b54b5b904b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:43.952088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6e5fb-b91d-4d13-8399-35b66dc7688c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:43.952371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6e5fb-b91d-4d13-8399-35b66dc7688c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:43.953699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6e5fb-b91d-4d13-8399-35b66dc7688c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:43.982129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-10b6e5fb-b91d-4d13-8399-35b66dc7688c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:43.983243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10b6e5fb-b91d-4d13-8399-35b66dc7688c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:44.026657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:44.036840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:44.049096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a7b20-4ad9-4353-915f-a8bca737b90a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.060572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.061299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.061576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.061845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.062748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.064524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.084475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 928/1865] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:44.093811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.096385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.098492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.106001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 938/1866] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:43 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:16:44.110564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.113393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:44.134996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d44d-4c46-4ea4-8699-09134f9cde00 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.140312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.141198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.141484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.142227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.143466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.145257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.154510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:44.155787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:44.159494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 939/1867] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:44.163017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.178983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8371743-c794-410b-8ce9-536677f54774 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.179992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 929/1868] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:44.186205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.188175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.198097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413345a0-2ce9-4e62-b8a3-f1e844b38e50 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.202249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.202636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.203078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.203860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.204398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.205254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.205623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.205993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.206871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:44.207284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.209088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.214776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.229734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 930/1869] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:44.233744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.246055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e955-0c5c-4be0-be2f-311895e025c6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.251426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 940/1870] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:44.252488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.255369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.256190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.256711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.257108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.258112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.261198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.278423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ff14f-65b0-4489-a296-44ea63b30642 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.294195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bea22f-4a72-4e1d-9a33-39c40ef219af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.295641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bea22f-4a72-4e1d-9a33-39c40ef219af admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.296075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bea22f-4a72-4e1d-9a33-39c40ef219af admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.296497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bea22f-4a72-4e1d-9a33-39c40ef219af admin admin] PATH_INFO: `/v3/projects/af2928ac8a8e4e3a8bc0b2fcc7c81265/users/eb2ccb70920e48e580d014b6eaad238a/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.319571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 931/1871] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:44 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:44.320995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.322309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bea22f-4a72-4e1d-9a33-39c40ef219af admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2928ac8a8e4e3a8bc0b2fcc7c81265, user_id=eb2ccb70920e48e580d014b6eaad238a, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.325213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bea22f-4a72-4e1d-9a33-39c40ef219af admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.358822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 941/1872] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:44 2022] PUT /identity/v3/projects/af2928ac8a8e4e3a8bc0b2fcc7c81265/users/eb2ccb70920e48e580d014b6eaad238a/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:44.370554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.370917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.371197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.431516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] MFA Rules not processed for user `eb2ccb70920e48e580d014b6eaad238a`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:44.432817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:44.464848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349cf6f-c37f-474f-8b67-03bd80a9ffd9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.557090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.557930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.558219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.558495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.559240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.562000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.564756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 942/1873] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:44.583291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.585183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:44.594073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 932/1874] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:44 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:44.598800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.620030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b593f9a-0ced-46b2-9bb6-7b59f4f190dd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.628909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.629893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.630353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.630793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.632681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.635659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.656267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 933/1875] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:44.657189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.668111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de828-f006-4b43-93b9-a0f650f5b796 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.673506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.674197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.674483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.674789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.675535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.677350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.681491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04683683-d773-4d8d-ae4b-538eee99914d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.692606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851b321-f346-4f42-9070-c1345c28232d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.693524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 934/1876] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:44.694412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:44.699519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851b321-f346-4f42-9070-c1345c28232d service glance] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.699701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851b321-f346-4f42-9070-c1345c28232d service glance] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.701326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851b321-f346-4f42-9070-c1345c28232d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.707905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0a28-d4f8-4310-b90e-20a2b32c3f9a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:44.719529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:44.720817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.721295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.721574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] PATH_INFO: `/v3/projects/54b30bbafafe4764b076c58274c59af6/users/bf100f423c444d9ea07b716fe6404a72/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.725798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:44.740252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851b321-f346-4f42-9070-c1345c28232d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.741883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851b321-f346-4f42-9070-c1345c28232d service glance] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.751415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b30bbafafe4764b076c58274c59af6, user_id=bf100f423c444d9ea07b716fe6404a72, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:44.752905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 943/1877] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:44.754237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:44.780031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 935/1878] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:44 2022] PUT /identity/v3/projects/54b30bbafafe4764b076c58274c59af6/users/bf100f423c444d9ea07b716fe6404a72/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:44.790819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:44.791176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:44.791533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:44.836061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] MFA Rules not processed for user `bf100f423c444d9ea07b716fe6404a72`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:44.837630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:44.868018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:44.951790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 944/1879] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:44.956507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.037980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2f541-c516-4fa8-a2a2-a0cc5fd76a5c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.043137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.043805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.044085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.044356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.045069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.047008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.075238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 936/1880] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:44 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:45.077508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.088992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d9fe8-7ffe-4b4c-a0a7-8a13c3adad08 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.152836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.153535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.153897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.154166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.154966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.157147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.179557 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 945/1881] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:45 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:45.184184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.196297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797fd1c-4ca0-4452-aa88-67b73c1225b3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.201166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.201870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.202158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.202433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.203248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.204950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.220499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 937/1882] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:16:45.226327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.239265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27a941-cccf-4604-be5b-fd0d16f30e12 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.244530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.245998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.245998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.246141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] PATH_INFO: `/v3/projects/b958e3a3440a4f65b65a0767dc376f3f/users/205c5b90d6db413996b73a948057c9d3/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.263738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b958e3a3440a4f65b65a0767dc376f3f, user_id=205c5b90d6db413996b73a948057c9d3, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.266571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.290988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 946/1883] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:45 2022] PUT /identity/v3/projects/b958e3a3440a4f65b65a0767dc376f3f/users/205c5b90d6db413996b73a948057c9d3/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:45.295363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.307091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733b6f-0056-4c4b-b6f2-a59eec00c3f8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.371658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c279d-f8b6-4aa8-871a-63b0a31186db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.372312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c279d-f8b6-4aa8-871a-63b0a31186db admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.372591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c279d-f8b6-4aa8-871a-63b0a31186db admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.372864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c279d-f8b6-4aa8-871a-63b0a31186db admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.373590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c279d-f8b6-4aa8-871a-63b0a31186db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.375231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c279d-f8b6-4aa8-871a-63b0a31186db admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.385580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 938/1884] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:16:45.390077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.401032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37b554-4915-465f-81bb-130b8467fd1e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.406948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.407833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.408116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.408390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] PATH_INFO: `/v3/projects/b958e3a3440a4f65b65a0767dc376f3f/users/205c5b90d6db413996b73a948057c9d3/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.410812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b958e3a3440a4f65b65a0767dc376f3f, user_id=205c5b90d6db413996b73a948057c9d3, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.413455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.426596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 947/1885] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:45 2022] PUT /identity/v3/projects/b958e3a3440a4f65b65a0767dc376f3f/users/205c5b90d6db413996b73a948057c9d3/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:45.437046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94afd9-d7a9-4591-90ec-d1bbe7624b8e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.437323 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94afd9-d7a9-4591-90ec-d1bbe7624b8e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.437593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94afd9-d7a9-4591-90ec-d1bbe7624b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.468813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6f94afd9-d7a9-4591-90ec-d1bbe7624b8e None None] MFA Rules not processed for user `205c5b90d6db413996b73a948057c9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:45.470969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6f94afd9-d7a9-4591-90ec-d1bbe7624b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:45.523105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:45.558248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 939/1886] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:45.569725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:45.572039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:45.631289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf7205-b49c-4e14-8f50-cc7292af8b36 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:45.635629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:45.636268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:45.636550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:45.636826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:45.663299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:45.699983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:45.699983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:45.708269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 948/1887] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:46.373059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:46.373616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:46.373949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:46.410510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:46.411887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:46.508044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 940/1888] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:16:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:46.523973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:46.532564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:46.757715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3b44-3f82-4ab1-a9ac-be6a0a245536 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:46.776385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f227-8ef9-46fc-b6a1-378fd5bced1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:46.777936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f227-8ef9-46fc-b6a1-378fd5bced1a service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:46.778409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f227-8ef9-46fc-b6a1-378fd5bced1a service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:46.778901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f227-8ef9-46fc-b6a1-378fd5bced1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:46.797355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f227-8ef9-46fc-b6a1-378fd5bced1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:46.799552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f227-8ef9-46fc-b6a1-378fd5bced1a service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:46.811202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 949/1889] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:16:48.528965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:48.530961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:48.607779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58eeb1-0f04-4335-8125-4a3977396f20 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:48.612271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:48.613148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:48.613420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:48.613687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] PATH_INFO: `/v3/users/0bc8e210bef54261a434c12a37bb1c2f` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:48.614915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc8e210bef54261a434c12a37bb1c2f)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:48.616884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:48.666873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc8e210bef54261a434c12a37bb1c2f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:48.681119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc8e210bef54261a434c12a37bb1c2f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:48.689124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc8e210bef54261a434c12a37bb1c2f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:48.709811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc8e210bef54261a434c12a37bb1c2f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:48.715532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc8e210bef54261a434c12a37bb1c2f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:48.726770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 941/1890] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:48 2022] DELETE /identity/v3/users/0bc8e210bef54261a434c12a37bb1c2f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:16:48.730600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:48.731158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:48.731243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:48.751601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:48.752899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:48.849003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 950/1891] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:48.859100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:48.860580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:48.865998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:48.931713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770833e-a9d8-4573-99d7-de7dc2910d82 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:48.935442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:48.936114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:48.936760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:48.936760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:48.947967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:48.949441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:48.964881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 942/1892] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:50.152322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:50.154544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:50.170354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf182d7-0867-413e-9ae4-3477199df90f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:50.177297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:50.178649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:50.179165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:50.179621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] PATH_INFO: `/v3/projects/29dd71f687a24c4eb04278fda56e523f` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:50.181587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=29dd71f687a24c4eb04278fda56e523f)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:50.185260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:50.299075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29dd71f687a24c4eb04278fda56e523f', 'request_id': 'req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.301844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:50.310289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29dd71f687a24c4eb04278fda56e523f', 'request_id': 'req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.322005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29dd71f687a24c4eb04278fda56e523f', 'request_id': 'req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.333890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] The token cache is being invalidate because project 29dd71f687a24c4eb04278fda56e523f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:50.333890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.334578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.338753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 951/1893] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:50 2022] DELETE /identity/v3/projects/29dd71f687a24c4eb04278fda56e523f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:50.341186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:50.355513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67491ac-e1a1-40d4-a604-230c4b3069eb service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:50.491336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:50.496049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:50.496525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:50.496925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] PATH_INFO: `/v3/users/3e23090c3c6f4be9ad3ff1d5379a6d55` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:50.500017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e23090c3c6f4be9ad3ff1d5379a6d55)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:50.503375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:50.534978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:50.536444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:50.625934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e23090c3c6f4be9ad3ff1d5379a6d55'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.648009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf42f95-b83c-4d96-9de7-9a6a5447ac6f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:50.659081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e23090c3c6f4be9ad3ff1d5379a6d55'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.663355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:50.667029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:50.667550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:50.667849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:50.669827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:50.671765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e23090c3c6f4be9ad3ff1d5379a6d55'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.680362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e23090c3c6f4be9ad3ff1d5379a6d55'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.681127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e23090c3c6f4be9ad3ff1d5379a6d55'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:50.690754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 943/1894] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:50 2022] DELETE /identity/v3/users/3e23090c3c6f4be9ad3ff1d5379a6d55 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:50.761454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:50.763113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd3dc0-c069-445c-b41e-37de9879ff7a service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:50.776273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 952/1895] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:51.376960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:51.379029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:51.421103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:51.469904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f0df-418f-4fb2-a423-56ee73ab5c8f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:51.475365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:51.476329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:51.476646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:51.477239 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] PATH_INFO: `/v3/projects/0947020c6daa42308f6ec5da4001ac77` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:51.479083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0947020c6daa42308f6ec5da4001ac77)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:51.481436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:51.550495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0947020c6daa42308f6ec5da4001ac77', 'request_id': 'req-706b7224-40cb-4216-8cf2-17931fbc4d97'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:51.557298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0947020c6daa42308f6ec5da4001ac77', 'request_id': 'req-706b7224-40cb-4216-8cf2-17931fbc4d97'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:51.567027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0947020c6daa42308f6ec5da4001ac77', 'request_id': 'req-706b7224-40cb-4216-8cf2-17931fbc4d97'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:51.586667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] The token cache is being invalidate because project 0947020c6daa42308f6ec5da4001ac77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:51.587339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:51.591064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-706b7224-40cb-4216-8cf2-17931fbc4d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:51.593795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 944/1896] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:51 2022] DELETE /identity/v3/projects/0947020c6daa42308f6ec5da4001ac77 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:51.598877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:51.600150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:51.600732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:51.632190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:51.632190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:51.701262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 953/1897] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:51.709318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:51.709505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:51.709827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:51.733183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:51.734423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:51.745827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 945/1898] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:51.750157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:51.761089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f487bd-3a9f-4baa-807a-a3ea6e3b0ee2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:51.766541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:51.767458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:51.767868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:51.768152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:51.769254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:51.769595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:51.771239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:51.789557 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 954/1899] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:16:51.790815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:51.803649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d1375-1eef-47ed-b542-9d06f58929d1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:51.817420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:51.818498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:51.819004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:51.819340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:51.820118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:51.822326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:51.877985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 946/1900] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:51 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:51.889477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:51.910984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fd139-8226-4baf-af39-dc7451e98895 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:51.935656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:52.000006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.000882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.001262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.001635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.002452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.004538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.030645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 955/1901] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:51 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:52.034256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.046254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1571f7-af7f-4334-ba72-55e6530c3f7a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.051924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.052879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.053196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.053744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.054665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.057820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.076668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 947/1902] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:52.077653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.091964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b884d6d-2330-464a-a79f-99b21fb02de5 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.098391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d6cab-ed47-4ae3-88cf-07c45ad15b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.099092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d6cab-ed47-4ae3-88cf-07c45ad15b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.099367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d6cab-ed47-4ae3-88cf-07c45ad15b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.099733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d6cab-ed47-4ae3-88cf-07c45ad15b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.100480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d6cab-ed47-4ae3-88cf-07c45ad15b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.103047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d6cab-ed47-4ae3-88cf-07c45ad15b63 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.132190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 956/1903] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:52.136805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.148684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9dfcc-23d8-4016-a978-2bfa03066045 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.154603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.156362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.156921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.159059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] PATH_INFO: `/v3/projects/a85c64ea436a4286843e72f0bfcce33a/users/b84f442ca0dc453b9cf6df19a1987000/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.197479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85c64ea436a4286843e72f0bfcce33a, user_id=b84f442ca0dc453b9cf6df19a1987000, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.201427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.237241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 948/1904] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:52 2022] PUT /identity/v3/projects/a85c64ea436a4286843e72f0bfcce33a/users/b84f442ca0dc453b9cf6df19a1987000/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:52.239417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.240375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.240668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.327922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] MFA Rules not processed for user `b84f442ca0dc453b9cf6df19a1987000`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:52.329190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:52.471723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 957/1905] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:52.475925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.574851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:52.591119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ba1c6-2ab2-4df6-bda8-6e82ae070fb6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.600359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.601074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.601356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.601633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.602478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.604610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.651233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 949/1906] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:52 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:16:52.652545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.672617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03815ab-ecde-4455-9f88-ef85237be29f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.801832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.804161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.804732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.805277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.806254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.809382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.839370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 958/1907] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:52 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:52.844366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.863891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec979e-a86a-4de2-91ef-895cc40aba61 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.869972 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.870808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.871160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.871435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.872179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.873791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.890351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 950/1908] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:52.901546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.914342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076273e-f9e3-4592-9aad-452f56b7080c admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:52.921959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:52.922718 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:52.923123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:52.924042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] PATH_INFO: `/v3/projects/53188c64feb74f6993dc7cd4367f5ae2/users/2f4ffe7c3d9f4412b94ce6cddb004dab/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:52.943890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:52.953304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=53188c64feb74f6993dc7cd4367f5ae2, user_id=2f4ffe7c3d9f4412b94ce6cddb004dab, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:52.956451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:52.978466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 959/1909] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:52 2022] PUT /identity/v3/projects/53188c64feb74f6993dc7cd4367f5ae2/users/2f4ffe7c3d9f4412b94ce6cddb004dab/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:52.979755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:52.993606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25741216-bd34-4cfc-9e5d-6b5d8c44f4e4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:53.093189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea38e78-392e-4919-b320-d6b488ca6335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:53.093888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea38e78-392e-4919-b320-d6b488ca6335 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:53.094171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea38e78-392e-4919-b320-d6b488ca6335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:53.094447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea38e78-392e-4919-b320-d6b488ca6335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:53.095246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea38e78-392e-4919-b320-d6b488ca6335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:53.096881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea38e78-392e-4919-b320-d6b488ca6335 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:53.108564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 951/1910] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:53.113378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:53.124708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8984c7-be6a-4532-9664-2549573192fa admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:53.129578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:53.130499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:53.130957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:53.131299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] PATH_INFO: `/v3/projects/53188c64feb74f6993dc7cd4367f5ae2/users/2f4ffe7c3d9f4412b94ce6cddb004dab/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:53.133519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53188c64feb74f6993dc7cd4367f5ae2, user_id=2f4ffe7c3d9f4412b94ce6cddb004dab, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:53.136251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:53.147175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 960/1911] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:53 2022] PUT /identity/v3/projects/53188c64feb74f6993dc7cd4367f5ae2/users/2f4ffe7c3d9f4412b94ce6cddb004dab/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:53.157680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:53.157937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:53.158426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:53.191494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] MFA Rules not processed for user `2f4ffe7c3d9f4412b94ce6cddb004dab`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:53.192783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:53.266076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 952/1912] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:53.277584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:53.279394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:53.349991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621c55f-2b4a-4841-80da-84aaf12f8e5d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:53.354636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf0e7f-ca77-4eb9-897f-4518bee015f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:53.355337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0e7f-ca77-4eb9-897f-4518bee015f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:53.355680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0e7f-ca77-4eb9-897f-4518bee015f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:53.355954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0e7f-ca77-4eb9-897f-4518bee015f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:53.372022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf0e7f-ca77-4eb9-897f-4518bee015f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:53.373476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf0e7f-ca77-4eb9-897f-4518bee015f6 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:53.387686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 961/1913] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:53.994108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:54.086873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:54.151126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9866c3c-764f-46bb-8fd5-d97210a26620 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:54.165058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:54.166864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:54.167343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:54.167769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] PATH_INFO: `/v3/users/b84f442ca0dc453b9cf6df19a1987000` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:54.170194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84f442ca0dc453b9cf6df19a1987000)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:54.175818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:54.319446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84f442ca0dc453b9cf6df19a1987000'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:54.363621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84f442ca0dc453b9cf6df19a1987000'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:54.379047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84f442ca0dc453b9cf6df19a1987000'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:54.405318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84f442ca0dc453b9cf6df19a1987000'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:54.407108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84f442ca0dc453b9cf6df19a1987000'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:54.416042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 953/1914] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:53 2022] DELETE /identity/v3/users/b84f442ca0dc453b9cf6df19a1987000 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:54.420387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:54.420983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:54.420983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:54.440076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:54.441112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:54.526992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 962/1915] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:54.536908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:54.538916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:54.642311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddff733-eb49-4ac2-92fa-b3da1c8f159e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:54.647386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:54.648784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:54.650338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:54.650860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:54.669339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:54.672168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:54.689538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 954/1916] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:55.427348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:55.429448 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:55.447711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cf261-3be3-405b-9c3d-19ffccb62a4c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:55.455065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:55.456202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:55.456583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:55.457101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] PATH_INFO: `/v3/projects/a85c64ea436a4286843e72f0bfcce33a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:55.458996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85c64ea436a4286843e72f0bfcce33a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:55.462340 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:55.528924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85c64ea436a4286843e72f0bfcce33a', 'request_id': 'req-3de83284-4b43-4293-aff5-94aa16766cd4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.535887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85c64ea436a4286843e72f0bfcce33a', 'request_id': 'req-3de83284-4b43-4293-aff5-94aa16766cd4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.545873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85c64ea436a4286843e72f0bfcce33a', 'request_id': 'req-3de83284-4b43-4293-aff5-94aa16766cd4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.559424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] The token cache is being invalidate because project a85c64ea436a4286843e72f0bfcce33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:55.559814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.560515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.563203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 963/1917] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:55 2022] DELETE /identity/v3/projects/a85c64ea436a4286843e72f0bfcce33a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:55.568721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:55.594818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943755a-d65e-435b-a01e-cd4c1212068a service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:55.708270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:55.708270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:55.708270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:55.708270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] PATH_INFO: `/v3/users/2f4ffe7c3d9f4412b94ce6cddb004dab` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:55.709521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4ffe7c3d9f4412b94ce6cddb004dab)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:55.711912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:55.734536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:55.788690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4ffe7c3d9f4412b94ce6cddb004dab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.804255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4ffe7c3d9f4412b94ce6cddb004dab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.813502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4ffe7c3d9f4412b94ce6cddb004dab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.820896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4ffe7c3d9f4412b94ce6cddb004dab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.821580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4ffe7c3d9f4412b94ce6cddb004dab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:55.827697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 955/1918] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:55 2022] DELETE /identity/v3/users/2f4ffe7c3d9f4412b94ce6cddb004dab => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:16:55.986471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:55.988074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:55.999259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:56.004215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de83284-4b43-4293-aff5-94aa16766cd4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:56.136161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:56.137072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.137359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.137687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] PATH_INFO: `/v3/users/bf100f423c444d9ea07b716fe6404a72` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.138993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf100f423c444d9ea07b716fe6404a72)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:56.141463 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:56.228403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf100f423c444d9ea07b716fe6404a72'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.245818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:56.247417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:56.253634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf100f423c444d9ea07b716fe6404a72'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.269111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf100f423c444d9ea07b716fe6404a72'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.270646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c918c1-6a15-44c1-b763-bf03ea401d77 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:56.279206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:56.279899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf100f423c444d9ea07b716fe6404a72'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.280315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.280580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.280699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-92c8d084-0135-4599-8e00-3757b919e536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf100f423c444d9ea07b716fe6404a72'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.280868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] PATH_INFO: `/v3/projects/53188c64feb74f6993dc7cd4367f5ae2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.282607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53188c64feb74f6993dc7cd4367f5ae2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:56.285559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:56.291358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 964/1919] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:55 2022] DELETE /identity/v3/users/bf100f423c444d9ea07b716fe6404a72 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:56.295754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.296550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.296550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.319251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:56.320557 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:56.407296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53188c64feb74f6993dc7cd4367f5ae2', 'request_id': 'req-0b409b2a-36eb-440c-a7ab-c9a4688a5653'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.416181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53188c64feb74f6993dc7cd4367f5ae2', 'request_id': 'req-0b409b2a-36eb-440c-a7ab-c9a4688a5653'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.432196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53188c64feb74f6993dc7cd4367f5ae2', 'request_id': 'req-0b409b2a-36eb-440c-a7ab-c9a4688a5653'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.465940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] The token cache is being invalidate because project 53188c64feb74f6993dc7cd4367f5ae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:56.467754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.468438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b409b2a-36eb-440c-a7ab-c9a4688a5653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:56.473574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 956/1920] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:56 2022] DELETE /identity/v3/projects/53188c64feb74f6993dc7cd4367f5ae2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:16:56.477116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61b4ee-08ff-4186-8050-c917b7b80bf9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.478674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61b4ee-08ff-4186-8050-c917b7b80bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.481064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61b4ee-08ff-4186-8050-c917b7b80bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.495225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 965/1921] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:56.506635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:56.510079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:56.532039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7f61b4ee-08ff-4186-8050-c917b7b80bf9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:56.534265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7f61b4ee-08ff-4186-8050-c917b7b80bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:56.556070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:56.611887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498506b0-3d7e-4ada-a7ae-22fa5d8bb187 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:56.653511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 957/1922] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:56.656900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.658723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.662125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.677267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:56.678035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.678435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.678901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.687406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:56.689482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:56.691501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:56.693078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:56.704048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 958/1923] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:56.705518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:56.708900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 966/1924] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:56.712404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:56.718317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5ef6-b6aa-41bb-abbc-c68bd13da73d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:56.728306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:56.730598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.731660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.732272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.734483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:56.735313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:56.737964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:56.755306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 959/1925] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:16:56.760507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:56.775207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0955-c631-41cf-9f75-c5784cf0efb1 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:56.782607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:56.783696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.784206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.784611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.785552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:56.788510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:56.842504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 967/1926] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:56 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:56.856678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:56.871528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b07d0a-5037-49ab-9189-536a9bd9c186 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:56.985918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd27d3b-2dcd-4ece-adcb-7fdb200f8c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:56.986938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd27d3b-2dcd-4ece-adcb-7fdb200f8c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:56.987377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd27d3b-2dcd-4ece-adcb-7fdb200f8c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:56.987778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd27d3b-2dcd-4ece-adcb-7fdb200f8c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:56.988497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd27d3b-2dcd-4ece-adcb-7fdb200f8c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:56.991505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd27d3b-2dcd-4ece-adcb-7fdb200f8c19 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.018737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 960/1927] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:56 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:57.024109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.035635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7d27-48f0-4991-a85e-e994c4ef0350 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.039481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e34ef-cab8-4b45-a23f-3ed1ffb99bc5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.040071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e34ef-cab8-4b45-a23f-3ed1ffb99bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.040626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e34ef-cab8-4b45-a23f-3ed1ffb99bc5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.042848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.043576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 961/1928] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:16:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 10:16:57.043784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.044043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.044325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.045066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.046807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.066101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b25b5-4836-4bd6-b6b1-1da8aa08c230 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.066476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b25b5-4836-4bd6-b6b1-1da8aa08c230 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.066676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b25b5-4836-4bd6-b6b1-1da8aa08c230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.068425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 968/1929] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:57.069972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.084602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d394a8-7cc2-4dcb-83ce-90079291a543 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.093114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.093816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.094099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.094461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.095390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.097092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.099873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5e0b25b5-4836-4bd6-b6b1-1da8aa08c230 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:57.103546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5e0b25b5-4836-4bd6-b6b1-1da8aa08c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:57.109877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 969/1930] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:57.114647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.132544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283e475-07b5-4b26-a641-88870a3903eb admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.138477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.139539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.139982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.140301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] PATH_INFO: `/v3/projects/996fa1aed3b84c4896698f32fb055fcd/users/64b4a6217e5b4cc3bbb6243ed236ddb3/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.235133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996fa1aed3b84c4896698f32fb055fcd, user_id=64b4a6217e5b4cc3bbb6243ed236ddb3, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.243662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.270864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 962/1931] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:57.282610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 970/1932] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:57 2022] PUT /identity/v3/projects/996fa1aed3b84c4896698f32fb055fcd/users/64b4a6217e5b4cc3bbb6243ed236ddb3/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:57.284793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.285137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.285417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.333750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.334726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] MFA Rules not processed for user `64b4a6217e5b4cc3bbb6243ed236ddb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:57.336346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:57.337057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:57.486559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f1501-1671-45fc-a918-83f8e019e234 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.488638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 963/1933] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:57.489697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.491197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:57.493484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.494239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.494589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.495168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.498396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:57.508175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.509962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.517702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:57.525266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 971/1934] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:57.526396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.640328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83b010-7558-42e5-bc72-54177ad155ee service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.647447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.648845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.649883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.650321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.651275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.653541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.668830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e883b-250c-4c52-b0c1-2d48cc56204e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.675251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.676458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.676749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.677030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] PATH_INFO: `/v3/projects/54b30bbafafe4764b076c58274c59af6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.678264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b30bbafafe4764b076c58274c59af6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.680714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.720719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 972/1935] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:57 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:57.724864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.814524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273e22f-81a8-43ac-98e7-62da72c79c0e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:57.826326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b30bbafafe4764b076c58274c59af6', 'request_id': 'req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:57.836405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b30bbafafe4764b076c58274c59af6', 'request_id': 'req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:57.852901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b30bbafafe4764b076c58274c59af6', 'request_id': 'req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:57.881338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] The token cache is being invalidate because project 54b30bbafafe4764b076c58274c59af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:57.882310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:57.887163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:57.892106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 964/1936] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:57 2022] DELETE /identity/v3/projects/54b30bbafafe4764b076c58274c59af6 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:57.901173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:57.903846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:57.956406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:57.957243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:57.958218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:57.958791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:57.959747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:57.964207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:57.997629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 973/1937] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:57 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:16:57.998947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.000875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:58.018149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675e1a-e8a4-400b-9da1-5455287df7fb admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:58.036364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:58.037258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.037715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.038088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.039398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:58.042640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:58.063725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 974/1938] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:58.065558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.080031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faa97d-be93-41aa-a240-ec860ecfd1dc admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:58.087444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:58.088364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.088840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.089068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] PATH_INFO: `/v3/projects/35aa949ee4ea4a82ab6987919f4f43f3/users/253fdc1fe71f40189c26f3f1d12c6935/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.101252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:58.117015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=35aa949ee4ea4a82ab6987919f4f43f3, user_id=253fdc1fe71f40189c26f3f1d12c6935, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:58.121018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:58.144955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 975/1939] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:58 2022] PUT /identity/v3/projects/35aa949ee4ea4a82ab6987919f4f43f3/users/253fdc1fe71f40189c26f3f1d12c6935/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:58.156261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.160509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978f3b-f0ce-44ae-b5f0-037ab0459a7d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:58.248344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:58.288609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:58.289770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.290329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.290659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] PATH_INFO: `/v3/users/205c5b90d6db413996b73a948057c9d3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.292341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=205c5b90d6db413996b73a948057c9d3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:58.295037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:58.298562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a16a89-5e55-4d13-9839-854f211a8d7b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:58.310061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:58.311450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.311934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.312402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.314333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:58.317950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:58.348575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 976/1940] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:16:58.349821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.368915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d806b-e5c9-466a-bc0c-ffef786bf7a9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:58.376968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf434ccb-1f29-465c-95a2-7442eb529cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:58.378464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434ccb-1f29-465c-95a2-7442eb529cab admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.379055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434ccb-1f29-465c-95a2-7442eb529cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.379409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434ccb-1f29-465c-95a2-7442eb529cab admin admin] PATH_INFO: `/v3/projects/35aa949ee4ea4a82ab6987919f4f43f3/users/253fdc1fe71f40189c26f3f1d12c6935/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.382889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf434ccb-1f29-465c-95a2-7442eb529cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=35aa949ee4ea4a82ab6987919f4f43f3, user_id=253fdc1fe71f40189c26f3f1d12c6935, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:58.388466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf434ccb-1f29-465c-95a2-7442eb529cab admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:58.390539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205c5b90d6db413996b73a948057c9d3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:58.408009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 977/1941] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:16:58 2022] PUT /identity/v3/projects/35aa949ee4ea4a82ab6987919f4f43f3/users/253fdc1fe71f40189c26f3f1d12c6935/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:58.414754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205c5b90d6db413996b73a948057c9d3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:58.425060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.425505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.425937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.455773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205c5b90d6db413996b73a948057c9d3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:58.476516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] MFA Rules not processed for user `253fdc1fe71f40189c26f3f1d12c6935`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:58.478891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:58.488254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205c5b90d6db413996b73a948057c9d3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:58.490559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205c5b90d6db413996b73a948057c9d3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:58.508611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 965/1942] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:57 2022] DELETE /identity/v3/users/205c5b90d6db413996b73a948057c9d3 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:58.587278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 978/1943] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:58.600569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.602246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:58.730301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0e66-7194-4ff4-b72e-6f1e68752a6e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:58.743652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:58.746047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:58.747046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:58.747636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:58.830130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:58.831650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:58.843226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 966/1944] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:16:58.934016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.935704 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:58.997781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:58.999354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.027394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac77b6-db14-45a5-a10b-3805cd91e1d3 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:59.032379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:59.033318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.033779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.034214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.037682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.047110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:59.067724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:59.079420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c042922-c6b8-4729-8041-8fe47fe8aea6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:59.176595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:59.179035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732a08e-a81d-4c75-b887-8a106e1045d5 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:59.191180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 967/1945] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:16:59.226420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:59.227521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.228142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.232701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] PATH_INFO: `/v3/projects/b958e3a3440a4f65b65a0767dc376f3f` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.233736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b958e3a3440a4f65b65a0767dc376f3f)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:59.237208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:59.319725 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b958e3a3440a4f65b65a0767dc376f3f', 'request_id': 'req-3d775598-589a-40e4-906b-f6c7df612f2c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.329871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b958e3a3440a4f65b65a0767dc376f3f', 'request_id': 'req-3d775598-589a-40e4-906b-f6c7df612f2c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.343048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b958e3a3440a4f65b65a0767dc376f3f', 'request_id': 'req-3d775598-589a-40e4-906b-f6c7df612f2c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.364842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] The token cache is being invalidate because project b958e3a3440a4f65b65a0767dc376f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:16:59.365117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.367149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3d775598-589a-40e4-906b-f6c7df612f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.369030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 979/1946] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:16:58 2022] DELETE /identity/v3/projects/b958e3a3440a4f65b65a0767dc376f3f => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:16:59.375870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.376162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.376443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.430810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:59.432302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.597807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 968/1947] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:59.608297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.608778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.609297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.635173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:59.636878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.648638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:59.649991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.652253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7166cc-49c8-475b-adc9-8aaebb7e0ff9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:59.657404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:59.658671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.659189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.659476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] PATH_INFO: `/v3/users/64b4a6217e5b4cc3bbb6243ed236ddb3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.661364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b4a6217e5b4cc3bbb6243ed236ddb3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:59.663274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:59.669227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 980/1948] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:16:59.671497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:59.683124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0fc5c-9e74-4029-8be2-a1bc17cdf4b3 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:59.688552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:59.689770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.690518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.697284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.698801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:16:59.699241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:59.701948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:59.732587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 981/1949] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:16:59.745002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:59.759576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13157d59-a750-48b2-be44-f312688b75bf admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:59.761800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b4a6217e5b4cc3bbb6243ed236ddb3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.764552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:16:59.765536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.765848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.766098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.767401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:16:59.768867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:16:59.781594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b4a6217e5b4cc3bbb6243ed236ddb3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.791539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b4a6217e5b4cc3bbb6243ed236ddb3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.801424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 982/1950] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:16:59 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:16:59.801662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b4a6217e5b4cc3bbb6243ed236ddb3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.802412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-997aa573-d0ce-426c-9168-d7965a514abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b4a6217e5b4cc3bbb6243ed236ddb3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:16:59.806738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:59.809907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 969/1951] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:16:59 2022] DELETE /identity/v3/users/64b4a6217e5b4cc3bbb6243ed236ddb3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:16:59.810039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.816851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:16:59.817642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:16:59.818926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:16:59.840975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:16:59.842854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:16:59.862330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:59.881890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:16:59.899645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbedf6-c8d2-47cb-8fff-6498ac8c0765 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:16:59.934380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 970/1952] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:16:59.946355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:16:59.948699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:00.015622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.016371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.016756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.017110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.017803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.020632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.036261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af5b1-8637-45b2-80cc-179419cb59a8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.040319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ece918-7488-4448-9161-341782e025ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.040988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ece918-7488-4448-9161-341782e025ec service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.041264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ece918-7488-4448-9161-341782e025ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.041628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ece918-7488-4448-9161-341782e025ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.046786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 983/1953] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:16:59 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:00.049476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.052577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ece918-7488-4448-9161-341782e025ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.054501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ece918-7488-4448-9161-341782e025ec service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.061640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7e82-1bf1-426c-826c-4f8adc637a84 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.067512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.068231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.068510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.068785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.069526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.069672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 971/1954] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:00.071408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.083694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 984/1955] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:00.088283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ece918-7488-4448-9161-341782e025ec service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.099622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ece918-7488-4448-9161-341782e025ec service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.105926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fd36-eb37-4f75-a6e8-43f560fb9658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.106616 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fd36-eb37-4f75-a6e8-43f560fb9658 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.106940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fd36-eb37-4f75-a6e8-43f560fb9658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.107220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fd36-eb37-4f75-a6e8-43f560fb9658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.107966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fd36-eb37-4f75-a6e8-43f560fb9658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.109602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fd36-eb37-4f75-a6e8-43f560fb9658 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.125459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 972/1956] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:00.128975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.142308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada85a7-f8c8-4c18-a194-754da8a911f3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.157793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.159543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.159834 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.160116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] PATH_INFO: `/v3/projects/aca77a9e3da1499393888a9b91f1a465/users/2a784f231c6a47a189b23516142cf768/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.177756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca77a9e3da1499393888a9b91f1a465, user_id=2a784f231c6a47a189b23516142cf768, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.181306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.202829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 985/1957] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:00 2022] PUT /identity/v3/projects/aca77a9e3da1499393888a9b91f1a465/users/2a784f231c6a47a189b23516142cf768/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:00.215368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.215824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.216669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.251581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] MFA Rules not processed for user `2a784f231c6a47a189b23516142cf768`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:00.253084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:00.346381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 973/1958] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:00.360553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.362568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:00.479405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7ed1e-187f-4108-bcd0-6e9e20f29ef4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.488426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.489280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.489402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.489702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.516430 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.518134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.523990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.526697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:00.543677 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 986/1959] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:00.571975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.677548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:00.716661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:00.723943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3020b7-1146-4983-b8d2-5097bc326bfd service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.732117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8031-7566-4f91-9b3a-ff65b2858ccf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:00.735825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.736734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.737009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.737279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] PATH_INFO: `/v3/users/2a784f231c6a47a189b23516142cf768` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.738816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a784f231c6a47a189b23516142cf768)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.741184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.747042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:00.747930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.748208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.748985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] PATH_INFO: `/v3/projects/996fa1aed3b84c4896698f32fb055fcd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.751265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996fa1aed3b84c4896698f32fb055fcd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:00.753295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:00.832105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996fa1aed3b84c4896698f32fb055fcd', 'request_id': 'req-921c0c45-d54f-4aec-bd21-4502654050e0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.840428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996fa1aed3b84c4896698f32fb055fcd', 'request_id': 'req-921c0c45-d54f-4aec-bd21-4502654050e0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.850815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a784f231c6a47a189b23516142cf768'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.857260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996fa1aed3b84c4896698f32fb055fcd', 'request_id': 'req-921c0c45-d54f-4aec-bd21-4502654050e0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.872669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a784f231c6a47a189b23516142cf768'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.882529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a784f231c6a47a189b23516142cf768'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.883912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] The token cache is being invalidate because project 996fa1aed3b84c4896698f32fb055fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:00.884207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.896148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.908919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 974/1960] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:00 2022] DELETE /identity/v3/projects/996fa1aed3b84c4896698f32fb055fcd => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:00.916598 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a784f231c6a47a189b23516142cf768'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.916873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:00.919799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fbb8f03a-3893-44b8-8900-3b7500cee689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a784f231c6a47a189b23516142cf768'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:00.920981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:00.938497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 987/1961] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:00 2022] DELETE /identity/v3/users/2a784f231c6a47a189b23516142cf768 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:00.944354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:00.944843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:00.945199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:00.991460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:00.993564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:01.024302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c0c45-d54f-4aec-bd21-4502654050e0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:01.069586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 988/1962] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:01.081896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:01.083520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:01.095976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:01.096950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:01.097328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:01.097580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] PATH_INFO: `/v3/users/253fdc1fe71f40189c26f3f1d12c6935` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:01.098920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253fdc1fe71f40189c26f3f1d12c6935)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:01.101340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:01.132606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:01.155252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816f4b-0b04-4f8c-9933-686ce5631e9d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:01.162200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:01.163005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:01.163856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:01.164205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:01.177552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253fdc1fe71f40189c26f3f1d12c6935'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:01.194594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:01.196999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:01.203188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253fdc1fe71f40189c26f3f1d12c6935'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:01.212074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 989/1963] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:01.212861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253fdc1fe71f40189c26f3f1d12c6935'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:01.222728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253fdc1fe71f40189c26f3f1d12c6935'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:01.223571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253fdc1fe71f40189c26f3f1d12c6935'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:01.232203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 975/1964] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:00 2022] DELETE /identity/v3/users/253fdc1fe71f40189c26f3f1d12c6935 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:01.792771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:01.794362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:01.832900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0417e-fd4a-464b-b1c0-3f6c4cbef83b service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:01.876147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:01.877568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:01.881163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:01.964254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:01.965249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:01.965602 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:01.965951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] PATH_INFO: `/v3/projects/aca77a9e3da1499393888a9b91f1a465` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:01.967625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca77a9e3da1499393888a9b91f1a465)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:01.969653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:01.978470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7505aeb-4135-4620-ab9f-37cc2006d0b5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:01.986437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:01.987477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:01.987789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:01.988089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] PATH_INFO: `/v3/projects/35aa949ee4ea4a82ab6987919f4f43f3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:01.989363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35aa949ee4ea4a82ab6987919f4f43f3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:01.991708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.031819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca77a9e3da1499393888a9b91f1a465', 'request_id': 'req-b0cbb964-59f6-4932-9146-0983015caece'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.038343 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca77a9e3da1499393888a9b91f1a465', 'request_id': 'req-b0cbb964-59f6-4932-9146-0983015caece'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.048712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35aa949ee4ea4a82ab6987919f4f43f3', 'request_id': 'req-5f0032de-b292-4fd2-aabd-d8aca723b387'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.049577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca77a9e3da1499393888a9b91f1a465', 'request_id': 'req-b0cbb964-59f6-4932-9146-0983015caece'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.054885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35aa949ee4ea4a82ab6987919f4f43f3', 'request_id': 'req-5f0032de-b292-4fd2-aabd-d8aca723b387'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.061850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35aa949ee4ea4a82ab6987919f4f43f3', 'request_id': 'req-5f0032de-b292-4fd2-aabd-d8aca723b387'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.069073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] The token cache is being invalidate because project aca77a9e3da1499393888a9b91f1a465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:02.069567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.070489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b0cbb964-59f6-4932-9146-0983015caece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.073153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 990/1965] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:01 2022] DELETE /identity/v3/projects/aca77a9e3da1499393888a9b91f1a465 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:02.078453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ccd3c-69be-4a06-af47-239b203ee189 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.078958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ccd3c-69be-4a06-af47-239b203ee189 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.079394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ccd3c-69be-4a06-af47-239b203ee189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.081714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] The token cache is being invalidate because project 35aa949ee4ea4a82ab6987919f4f43f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:02.082075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.083137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f0032de-b292-4fd2-aabd-d8aca723b387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:02.085252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 976/1966] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:01 2022] DELETE /identity/v3/projects/35aa949ee4ea4a82ab6987919f4f43f3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:02.092326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ff80-bdc1-4c71-a12e-af765abf2052 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.092642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ff80-bdc1-4c71-a12e-af765abf2052 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.093046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ff80-bdc1-4c71-a12e-af765abf2052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.106809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8e0ccd3c-69be-4a06-af47-239b203ee189 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:02.108079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8e0ccd3c-69be-4a06-af47-239b203ee189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:02.114041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a044ff80-bdc1-4c71-a12e-af765abf2052 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:02.115173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a044ff80-bdc1-4c71-a12e-af765abf2052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:02.148876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:02.199113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 991/1967] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:02.207109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 977/1968] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:02.208213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.208647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.209072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.215497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.215853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.216405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.233071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:02.234658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:02.242866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:02.244256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:02.247848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 992/1969] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:02.249292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.256306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 978/1970] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:02.260945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.270386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce87e3-6942-4aaa-a33c-f5e6b2929154 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.272496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585caf24-f102-4a79-ab72-4dff25f0e61a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.276643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.277547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.277974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.278363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.279253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:02.279446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.279754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.280408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.280697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.280974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.281720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:02.281988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.282097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.283667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.302661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 993/1971] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:17:02.305401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.312205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 979/1972] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:02.317482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.328021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f169c2-db67-42a5-9806-71ea7ad55345 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.331742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288b42-7a8e-4c46-96a8-4daebe0399f2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.337578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.339171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.339457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.339683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.339806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.340514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.340976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.341692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.342040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.342851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.343209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.350935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.383637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 980/1973] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:02.388643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.429289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 994/1974] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:02.432840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.444307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:02.477450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6671b-c445-45a5-bf1a-450841178773 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.554866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf075-7e66-4346-bab7-3436fac5e570 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.555464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.556489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.556972 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.565718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.565718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.565718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.569885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.570671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.570995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.571877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.572595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.574744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.593645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 981/1975] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:02.605021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.614915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 995/1976] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:02.616404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.630879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9c217-cdf9-4d86-bd84-a9eab558d33b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.640508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7f6d4-d480-42ad-8322-0263293d8a8c admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.654834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.655531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.655823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.656106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.657215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.659304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.659426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.661899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.662354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.670809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.678354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.683389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.690071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 982/1977] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:02.701935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.718962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 996/1978] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:02.727110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.743155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027f64-c3d5-43f0-98e2-17853ea1b5d8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.748298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071a7c-65c2-45f2-b314-1e5ff0cab039 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.752348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.753036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.753930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.754216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.754991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.757361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.757896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.758176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.758995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.759223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.760018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.761670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.782332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 983/1979] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:02.787877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.788938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 997/1980] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:02.794404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:02.800018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:02.818170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f1d23-85e9-4bc4-9c17-0fabb99bb0fd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.834875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929f68-235a-4518-b87f-e63eb30f6ce8 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:02.835118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65a4b2-11c4-44d4-a0bc-55887c3ab423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.835118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a4b2-11c4-44d4-a0bc-55887c3ab423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.835118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a4b2-11c4-44d4-a0bc-55887c3ab423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.835118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65a4b2-11c4-44d4-a0bc-55887c3ab423 admin admin] PATH_INFO: `/v3/projects/b99acc778f9b42558f2952a8d0cd618e/users/d36f7d6d2d3e44db9ce3e44ad691836d/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.862965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d70cd1-7168-4dcd-aac6-14ceb63109cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:02.863992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70cd1-7168-4dcd-aac6-14ceb63109cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.864282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70cd1-7168-4dcd-aac6-14ceb63109cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.864564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70cd1-7168-4dcd-aac6-14ceb63109cd admin admin] PATH_INFO: `/v3/projects/f389b844cee34312a4c6640284c53ef0/users/b4f0a8c23ede41659a3963037e738219/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:02.880604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65a4b2-11c4-44d4-a0bc-55887c3ab423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99acc778f9b42558f2952a8d0cd618e, user_id=d36f7d6d2d3e44db9ce3e44ad691836d, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.883883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65a4b2-11c4-44d4-a0bc-55887c3ab423 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.927607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d70cd1-7168-4dcd-aac6-14ceb63109cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f389b844cee34312a4c6640284c53ef0, user_id=b4f0a8c23ede41659a3963037e738219, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:02.942731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 984/1981] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:02 2022] PUT /identity/v3/projects/b99acc778f9b42558f2952a8d0cd618e/users/d36f7d6d2d3e44db9ce3e44ad691836d/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:02.942953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d70cd1-7168-4dcd-aac6-14ceb63109cd admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:02.955402 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:02.955644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:02.955993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:03.010240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 998/1982] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:02 2022] PUT /identity/v3/projects/f389b844cee34312a4c6640284c53ef0/users/b4f0a8c23ede41659a3963037e738219/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:03.022963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:03.027268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:03.027841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:03.038942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:03.040302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:03.107947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] MFA Rules not processed for user `b4f0a8c23ede41659a3963037e738219`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:03.111125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:03.230053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 985/1983] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:03.247267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:03.251221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:03.311851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 999/1984] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:03.332446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:03.337073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:03.403047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69e953-6663-4188-bcbf-fe2dfe9e5765 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:03.409587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30810d39-5515-4585-a1e3-72e19bd2da1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:03.410485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810d39-5515-4585-a1e3-72e19bd2da1b service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:03.410925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810d39-5515-4585-a1e3-72e19bd2da1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:03.411249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30810d39-5515-4585-a1e3-72e19bd2da1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:03.428904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810d39-5515-4585-a1e3-72e19bd2da1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:03.432158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30810d39-5515-4585-a1e3-72e19bd2da1b service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:03.444960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 986/1985] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:17:03.540836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:03.572861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0680d-0bfc-4c7e-a745-c3d21d750b67 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:03.577645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:03.578963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:03.579349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:03.579986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:03.601896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:03.604404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:03.621297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1000/1986] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:04.557608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:04.557608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:04.557608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:04.619929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] MFA Rules not processed for user `d36f7d6d2d3e44db9ce3e44ad691836d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:04.621564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:04.701026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 987/1987] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:04.709087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:04.719319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a887969-45e1-46f6-a92c-a6be1ed1313c service glance] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:04.724437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:04.725139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:04.725457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:04.725728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:04.726777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:04.729151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:04.764713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1001/1988] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:04 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:04.769252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:04.781968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fc560-62fb-4cad-bb18-e03b00e0138a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:04.792157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:04.879394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:04.880144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:04.880454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:04.880727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:04.881452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:04.883681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:04.926274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 988/1989] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:04 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:04.927685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:04.946424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5f80d-1fc1-40ef-937f-714eb2174618 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:04.953278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:04.955531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:04.955871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:04.956320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:04.957273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:04.959608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:04.980355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1002/1990] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:04.984801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:04.996555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6f14-6d49-4863-bcb2-923acd76d716 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:05.001688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:05.002892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:05.003210 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:05.003662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] PATH_INFO: `/v3/projects/2d1fc6ec94c24a87a6ea7549572a960a/users/e898bafffff740a7b0eb10fb6a66a54d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:05.022271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1fc6ec94c24a87a6ea7549572a960a, user_id=e898bafffff740a7b0eb10fb6a66a54d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:05.025066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:05.039116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 989/1991] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:04 2022] PUT /identity/v3/projects/2d1fc6ec94c24a87a6ea7549572a960a/users/e898bafffff740a7b0eb10fb6a66a54d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:05.043848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:05.054272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2429f40-3fd2-4a6e-8a27-5938d3e04af9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:05.117406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f7167-b0c9-42f7-b600-c81e12ceb02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:05.118007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f7167-b0c9-42f7-b600-c81e12ceb02d admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:05.118284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f7167-b0c9-42f7-b600-c81e12ceb02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:05.118555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f7167-b0c9-42f7-b600-c81e12ceb02d admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:05.119316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f7167-b0c9-42f7-b600-c81e12ceb02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:05.120913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f7167-b0c9-42f7-b600-c81e12ceb02d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:05.130924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1003/1992] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:05.135859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:05.146400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6276f4-6025-4819-8d62-669a279c1526 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:05.151878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:05.153020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:05.153332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:05.153604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] PATH_INFO: `/v3/projects/2d1fc6ec94c24a87a6ea7549572a960a/users/e898bafffff740a7b0eb10fb6a66a54d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:05.155763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1fc6ec94c24a87a6ea7549572a960a, user_id=e898bafffff740a7b0eb10fb6a66a54d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:05.158503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:05.169799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 990/1993] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:05 2022] PUT /identity/v3/projects/2d1fc6ec94c24a87a6ea7549572a960a/users/e898bafffff740a7b0eb10fb6a66a54d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:05.967016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:05.967617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:05.967617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:06.001146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] MFA Rules not processed for user `e898bafffff740a7b0eb10fb6a66a54d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:06.002748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:06.105332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1004/1994] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:06.118192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:06.120257 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:06.185337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1ac3c-f5b8-4c1d-88ec-ab1bd55c8c16 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:06.189307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:06.190029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:06.190342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:06.190606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:06.256140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:06.257678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:06.268319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 991/1995] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:17:10.880362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:10.881861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:10.896075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaffb18-5beb-4a17-b76f-a247a59af4bc None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:10.899890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a2ba1-20dd-4c83-a679-297a67241d75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:10.900534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a2ba1-20dd-4c83-a679-297a67241d75 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:10.900807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a2ba1-20dd-4c83-a679-297a67241d75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:10.901071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a2ba1-20dd-4c83-a679-297a67241d75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:10.912043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a2ba1-20dd-4c83-a679-297a67241d75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:10.913507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a2ba1-20dd-4c83-a679-297a67241d75 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:10.924413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1005/1996] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:10 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:16.263612 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:20.919707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:24.514215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:24.522101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:24.607559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e31c8-e0e5-49d9-b20e-e05308689d6f service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:24.612529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:24.614129 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:24.614414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:24.614724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] PATH_INFO: `/v3/users/2934818551254f8093086dd94eef5e08` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:24.616122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2934818551254f8093086dd94eef5e08)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:24.618152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:24.686193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2934818551254f8093086dd94eef5e08'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:24.712540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2934818551254f8093086dd94eef5e08'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:24.725886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2934818551254f8093086dd94eef5e08'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:24.743807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2934818551254f8093086dd94eef5e08'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:24.746206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2934818551254f8093086dd94eef5e08'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:24.755445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 992/1997] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:24 2022] DELETE /identity/v3/users/2934818551254f8093086dd94eef5e08 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:24.760047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:24.760579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:24.760754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:24.785506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:24.786972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:24.883947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1006/1998] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:24.897943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:24.900295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:24.998799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e8bf5-869b-47d3-8fb3-9691ee133aaa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:25.002442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:25.003629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:25.003909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:25.004180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:25.018148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:25.020354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:25.034781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 993/1999] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:25.339024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:25.340660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:25.355524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607378e-29d3-4461-bacc-8b0b9aeb5d6b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:25.367022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:25.367541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:25.367541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:25.367810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] PATH_INFO: `/v3/projects/a8c521bf29f14b9db28816bed9a4d0f3` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:25.369113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c521bf29f14b9db28816bed9a4d0f3)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:25.371093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:25.428066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c521bf29f14b9db28816bed9a4d0f3', 'request_id': 'req-cc162336-456d-422d-97ea-95cbcb26ae8f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.436053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c521bf29f14b9db28816bed9a4d0f3', 'request_id': 'req-cc162336-456d-422d-97ea-95cbcb26ae8f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.446244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c521bf29f14b9db28816bed9a4d0f3', 'request_id': 'req-cc162336-456d-422d-97ea-95cbcb26ae8f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.457672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] The token cache is being invalidate because project a8c521bf29f14b9db28816bed9a4d0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:25.457961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.458606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.460271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1007/2000] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:25 2022] DELETE /identity/v3/projects/a8c521bf29f14b9db28816bed9a4d0f3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:25.466529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:25.478306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd19fdf-95c2-4d67-891f-212c6f3db9b8 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:25.500876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:25.545093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:25.545955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:25.546236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:25.546505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] PATH_INFO: `/v3/users/8117614a253346bda80dee254726fdd6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:25.547941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8117614a253346bda80dee254726fdd6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:25.549889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:25.610666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8117614a253346bda80dee254726fdd6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.627700 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8117614a253346bda80dee254726fdd6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.640099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8117614a253346bda80dee254726fdd6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.648890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8117614a253346bda80dee254726fdd6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.649634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-94553ff2-7d17-44d9-8fff-7ba12aa93c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8117614a253346bda80dee254726fdd6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:25.656434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 994/2001] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:25 2022] DELETE /identity/v3/users/8117614a253346bda80dee254726fdd6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:25.976764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:25.978919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:25.997326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc162336-456d-422d-97ea-95cbcb26ae8f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.086052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.087022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.087308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.087562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] PATH_INFO: `/v3/projects/7415a5010971428a9582b84bf6af8376` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.088798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7415a5010971428a9582b84bf6af8376)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.091153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.151045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:26.191348 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7415a5010971428a9582b84bf6af8376', 'request_id': 'req-6f761634-07d6-4462-b33d-8ec915148094'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:26.199639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7415a5010971428a9582b84bf6af8376', 'request_id': 'req-6f761634-07d6-4462-b33d-8ec915148094'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:26.213779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7415a5010971428a9582b84bf6af8376', 'request_id': 'req-6f761634-07d6-4462-b33d-8ec915148094'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:26.231597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] The token cache is being invalidate because project 7415a5010971428a9582b84bf6af8376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:26.231597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:26.231597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6f761634-07d6-4462-b33d-8ec915148094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:26.231597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1008/2002] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:25 2022] DELETE /identity/v3/projects/7415a5010971428a9582b84bf6af8376 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:26.235941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.236587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.237167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.258063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:26.259487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:26.357884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 995/2003] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:26.366350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.366727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.367023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.392107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:26.393753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:26.402827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1009/2004] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:26.408709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:26.420420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57be3-2eea-4707-a9f7-f4f3f19484fc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.426175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.426912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.427186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.427455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.428236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:26.428535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.430597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.444730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 996/2005] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:26.450868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:26.463682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32d8f9-fe36-4d0f-843b-36e0ecf0d12c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.470380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.471125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.471411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.471717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.472698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.474586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.522911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1010/2006] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:26 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:26.527813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:26.539509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b995a98-0b6a-4f43-becf-b894651d4715 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.584181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:26.647082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.648085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.648544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.648968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.649833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.652384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.681501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 997/2007] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:26 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:26.685945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:26.709337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560991d-195b-42f1-9458-0864cbc758ea admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.724089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.725318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.725478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.725846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.726726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.729194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.742012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1011/2008] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:26.745995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:26.756998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7b3e9-1d18-407f-812f-d91be3c5b752 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.763346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00a307-613b-40c3-ad3a-792f51e81933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.764201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00a307-613b-40c3-ad3a-792f51e81933 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.764570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00a307-613b-40c3-ad3a-792f51e81933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.764951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00a307-613b-40c3-ad3a-792f51e81933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.765802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00a307-613b-40c3-ad3a-792f51e81933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.768534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00a307-613b-40c3-ad3a-792f51e81933 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.780495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 998/2009] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:26.790741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:26.810008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142ba38-ca86-48e6-b2bb-60aae284c82d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:26.819515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:26.820380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.820712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.820917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] PATH_INFO: `/v3/projects/2d9194371a964aa7b7c2d0c59dbfc259/users/3ddabb4633d44d5daff850dca268099d/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.839345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9194371a964aa7b7c2d0c59dbfc259, user_id=3ddabb4633d44d5daff850dca268099d, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:26.842857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:26.873928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1012/2010] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:26 2022] PUT /identity/v3/projects/2d9194371a964aa7b7c2d0c59dbfc259/users/3ddabb4633d44d5daff850dca268099d/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:26.888685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:26.889110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:26.889594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:26.926456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] MFA Rules not processed for user `3ddabb4633d44d5daff850dca268099d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:26.928245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:27.023421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 999/2011] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:27.024858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.133591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64438ca-c017-4323-a350-1388cab269cc admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.137191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.137774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.138210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.138380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.139180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.141368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.229876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1013/2012] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:27 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:27.234347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.266294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdb308-bc4b-4787-aa0b-67cc0d1330f4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.352946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.353660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.353943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.354234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.354992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.356959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.379586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1000/2013] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:27 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:27.383753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.394593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc328-4420-4231-90e1-450a983dbbf9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.399723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.400284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.400567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.400846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.401592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.403274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.420728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1014/2014] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:27.426685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.438649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d927d5-b7c8-4fa8-97ce-e74f830cfcff admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.443816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.444734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.445012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.445286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] PATH_INFO: `/v3/projects/c8aed8644e3d4956b56b189336cd2d6f/users/bb43958cfca644f8abf941924ff8ad7d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.461931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8aed8644e3d4956b56b189336cd2d6f, user_id=bb43958cfca644f8abf941924ff8ad7d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.464695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.488973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1001/2015] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:27 2022] PUT /identity/v3/projects/c8aed8644e3d4956b56b189336cd2d6f/users/bb43958cfca644f8abf941924ff8ad7d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:27.490943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.505128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c17972-b04b-4086-85be-7e9f40a60c0a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.541762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:27.582243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae22c36-ea12-44db-bab8-fb63a8d3f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.583173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae22c36-ea12-44db-bab8-fb63a8d3f346 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.583464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae22c36-ea12-44db-bab8-fb63a8d3f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.583745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae22c36-ea12-44db-bab8-fb63a8d3f346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.584489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae22c36-ea12-44db-bab8-fb63a8d3f346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.586205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae22c36-ea12-44db-bab8-fb63a8d3f346 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.602831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1015/2016] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:27.606848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.615598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:27.622881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efcd69-5a18-4f1f-a86b-05487b3cc970 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.634047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.635014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.635309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.636357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] PATH_INFO: `/v3/projects/c8aed8644e3d4956b56b189336cd2d6f/users/bb43958cfca644f8abf941924ff8ad7d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.640835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8aed8644e3d4956b56b189336cd2d6f, user_id=bb43958cfca644f8abf941924ff8ad7d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.648350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.680993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1002/2017] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:27 2022] PUT /identity/v3/projects/c8aed8644e3d4956b56b189336cd2d6f/users/bb43958cfca644f8abf941924ff8ad7d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:27.694310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.694310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.694800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.748372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] MFA Rules not processed for user `bb43958cfca644f8abf941924ff8ad7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:27.749870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:27.803208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.805500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:27.866864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1016/2018] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:27.886087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:27.891851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:27.895507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741993a-9bda-4a81-adf3-31516f2fecb9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.900605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.901513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.901800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.902078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] PATH_INFO: `/v3/users/d36f7d6d2d3e44db9ce3e44ad691836d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.903477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36f7d6d2d3e44db9ce3e44ad691836d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.905322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:27.969100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36f7d6d2d3e44db9ce3e44ad691836d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:27.981571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dc2f4-a630-4258-b36a-e51004ab1fcb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:27.984349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36f7d6d2d3e44db9ce3e44ad691836d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:27.985603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:27.986282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:27.986578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:27.986885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:27.996625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36f7d6d2d3e44db9ce3e44ad691836d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:27.998161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:27.999808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:28.008563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36f7d6d2d3e44db9ce3e44ad691836d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.011933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36f7d6d2d3e44db9ce3e44ad691836d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.018981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1017/2019] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:28.022690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1003/2020] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:27 2022] DELETE /identity/v3/users/d36f7d6d2d3e44db9ce3e44ad691836d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:28.136562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:28.145823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37746-9fe6-4820-8241-007c4c1d7378 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:28.210868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:28.223477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:28.224285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:28.224573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:28.224900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:28.227135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:28.262341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:28.263922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:28.321032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:28.322048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:28.337627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1018/2021] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:28.422415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c449e-056c-44bc-b93c-6815899e3954 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:28.429078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:28.435407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:28.435407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:28.435407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] PATH_INFO: `/v3/projects/b99acc778f9b42558f2952a8d0cd618e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:28.437804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99acc778f9b42558f2952a8d0cd618e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:28.440239 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:28.455136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:28.528723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99acc778f9b42558f2952a8d0cd618e', 'request_id': 'req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.535933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99acc778f9b42558f2952a8d0cd618e', 'request_id': 'req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.543224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99acc778f9b42558f2952a8d0cd618e', 'request_id': 'req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.563613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] The token cache is being invalidate because project b99acc778f9b42558f2952a8d0cd618e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:28.563902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.564557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.568100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1004/2022] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:28 2022] DELETE /identity/v3/projects/b99acc778f9b42558f2952a8d0cd618e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:28.572516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:28.584220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bca90-5760-4241-870f-6cff6289bb56 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:28.664098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:28.665409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:28.665850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:28.666262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] PATH_INFO: `/v3/users/e898bafffff740a7b0eb10fb6a66a54d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:28.668335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e898bafffff740a7b0eb10fb6a66a54d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:28.670556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:28.761226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e898bafffff740a7b0eb10fb6a66a54d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.782310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e898bafffff740a7b0eb10fb6a66a54d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.791887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e898bafffff740a7b0eb10fb6a66a54d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.800629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e898bafffff740a7b0eb10fb6a66a54d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.801379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ee4497a1-0b90-439d-95c3-fa87f8826de2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e898bafffff740a7b0eb10fb6a66a54d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:28.808038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1019/2023] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:28 2022] DELETE /identity/v3/users/e898bafffff740a7b0eb10fb6a66a54d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:29.007364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.013009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:29.029258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3e68c-5be6-453d-ab2a-c7b3198e56a4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.142390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.143365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.143760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.144036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] PATH_INFO: `/v3/projects/2d1fc6ec94c24a87a6ea7549572a960a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.145484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1fc6ec94c24a87a6ea7549572a960a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.147490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.211968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1fc6ec94c24a87a6ea7549572a960a', 'request_id': 'req-2813feed-cbae-437f-9761-0f3ebd710328'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:29.220698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1fc6ec94c24a87a6ea7549572a960a', 'request_id': 'req-2813feed-cbae-437f-9761-0f3ebd710328'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:29.227692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1fc6ec94c24a87a6ea7549572a960a', 'request_id': 'req-2813feed-cbae-437f-9761-0f3ebd710328'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:29.244552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] The token cache is being invalidate because project 2d1fc6ec94c24a87a6ea7549572a960a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:29.244843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:29.245584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2813feed-cbae-437f-9761-0f3ebd710328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:29.247344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1005/2024] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:29 2022] DELETE /identity/v3/projects/2d1fc6ec94c24a87a6ea7549572a960a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:29.252401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.252894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.253083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.271496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:29.272717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:29.297081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:29.348649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1020/2025] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:29.357283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.357574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.357962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.378456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:29.380250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:29.391881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1006/2026] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:29.397996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.410568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9d86-03c4-4c6e-bbc2-beb49706cf14 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.415437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.416123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.416411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.416691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.417463 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:29.417775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.419464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.432632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1021/2027] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:29.437265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.448820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f04db4-ce1f-4db7-8faa-c5551b9c674c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.454586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.455290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.455575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.455851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.456824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.459084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.492542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:29.501066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1007/2028] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:29 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:29.505578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.516896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245872cc-1abd-4224-a9db-8792b85aa307 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.588836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.589579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.589876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.590107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.591229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.593299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.623367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1022/2029] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:29 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:29.627794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.641377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeae719-1e28-46ed-92af-3795f290224c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.647341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.648032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.648541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.648813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.649591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.651278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.667120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1008/2030] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:29.671513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.683379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb136d-aada-46f3-923f-0217b19d187d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.702948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32889dc3-46f9-4b01-9f1b-41237e8297e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.704309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32889dc3-46f9-4b01-9f1b-41237e8297e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.704613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32889dc3-46f9-4b01-9f1b-41237e8297e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.704888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32889dc3-46f9-4b01-9f1b-41237e8297e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.705638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32889dc3-46f9-4b01-9f1b-41237e8297e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.707388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32889dc3-46f9-4b01-9f1b-41237e8297e9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.726905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1023/2031] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:29.730807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.742311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ded64-add8-44c7-90f5-b61659e89655 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:29.747234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:29.748131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.748411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.748718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] PATH_INFO: `/v3/projects/2296bbe541c949c8a5383a7ac8f98c4b/users/d8b1c1ac93024141a573d582f710d9d9/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.766069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2296bbe541c949c8a5383a7ac8f98c4b, user_id=d8b1c1ac93024141a573d582f710d9d9, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:29.769103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:29.791581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1009/2032] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:29 2022] PUT /identity/v3/projects/2296bbe541c949c8a5383a7ac8f98c4b/users/d8b1c1ac93024141a573d582f710d9d9/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:29.805518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:29.805893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:29.807028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:29.863794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] MFA Rules not processed for user `d8b1c1ac93024141a573d582f710d9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:29.865183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:29.963367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1024/2033] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:29.974592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:29.976386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:30.076281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93dc9-e9b0-4d35-b016-83003caea869 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:30.086999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:30.087845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:30.088132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:30.088409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:30.104205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:30.105669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:30.117415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1010/2034] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:33.944521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:33.946958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:33.974175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:34.035810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca713c6-7de2-4f59-822f-d6cb176fabe5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.040774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.041805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.042215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.042493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] PATH_INFO: `/v3/users/fe9a62b8fcdc400b9ceb993bb8f936e0` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.043823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9a62b8fcdc400b9ceb993bb8f936e0)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.045986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:34.103314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9a62b8fcdc400b9ceb993bb8f936e0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.115339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9a62b8fcdc400b9ceb993bb8f936e0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.129971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9a62b8fcdc400b9ceb993bb8f936e0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.140358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9a62b8fcdc400b9ceb993bb8f936e0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.141396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9ec91a-0de2-475f-ae00-ef0012d361ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9a62b8fcdc400b9ceb993bb8f936e0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.150003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1025/2035] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:33 2022] DELETE /identity/v3/users/fe9a62b8fcdc400b9ceb993bb8f936e0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:34.366926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:34.378951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df5a5e-243b-4d09-a74b-88b5ecbef5c1 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.445424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.446304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.446583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.446892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] PATH_INFO: `/v3/projects/54baeb39b08b4d69aac8c4d42043112a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.448162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54baeb39b08b4d69aac8c4d42043112a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.450092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:34.504167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54baeb39b08b4d69aac8c4d42043112a', 'request_id': 'req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.510936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54baeb39b08b4d69aac8c4d42043112a', 'request_id': 'req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.517787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54baeb39b08b4d69aac8c4d42043112a', 'request_id': 'req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.527914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:34.533744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] The token cache is being invalidate because project 54baeb39b08b4d69aac8c4d42043112a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:34.534368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.535533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5e07bc8-d9b0-44d2-8a1a-f661b88b3194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:34.537896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1011/2036] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:34 2022] DELETE /identity/v3/projects/54baeb39b08b4d69aac8c4d42043112a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:34.543650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.543938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.544214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.564678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:34.566273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:34.640199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1026/2037] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:34.648540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.648831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.649109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.671305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:34.672534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:34.690242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1012/2038] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:34.695262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:34.708986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a1d22-7818-49ff-8c70-9ab5d49adb7f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.713978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.714896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.715150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.715428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.716547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:34.716852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.718481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:34.731530 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1027/2039] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:34.736403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:34.747031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc11b-9d73-4e65-a698-b08d8d956347 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.753920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.754591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.754920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.755198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.755911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.757850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:34.788255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1013/2040] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:34 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:34.792855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:34.803978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2e2c-e0f6-4bcf-954b-85266125765b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.827197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:34.889291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.889982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.890420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.890558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.891340 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.893355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:34.918252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1028/2041] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:34 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:34.922524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:34.935791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d3691-2f7f-4cb6-a81f-29ace09c27b5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.940686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.942284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.942284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.942284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.942657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.944537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:34.956166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1014/2042] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:34.961432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:34.975887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b940849-ec96-47c1-a929-08b6eb0800b7 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:34.980937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5c553-0662-46b3-9107-1b20c7414984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:34.981627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c553-0662-46b3-9107-1b20c7414984 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:34.981909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c553-0662-46b3-9107-1b20c7414984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:34.982190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c553-0662-46b3-9107-1b20c7414984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:34.983087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5c553-0662-46b3-9107-1b20c7414984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:34.984963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5c553-0662-46b3-9107-1b20c7414984 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:35.002102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1029/2043] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:35.004332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:35.015577 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8414f0-0c99-4074-8c43-f9af5281f411 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:35.021103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:35.022016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:35.022321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:35.022599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] PATH_INFO: `/v3/projects/3d4512d231694fa187ada8905222baf7/users/50944673799940368982b7c196eb65b3/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:35.040106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4512d231694fa187ada8905222baf7, user_id=50944673799940368982b7c196eb65b3, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:35.042967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:35.060966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1015/2044] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:35 2022] PUT /identity/v3/projects/3d4512d231694fa187ada8905222baf7/users/50944673799940368982b7c196eb65b3/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:35.071712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:35.072542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:35.072835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:35.114143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] MFA Rules not processed for user `50944673799940368982b7c196eb65b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:35.115806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:35.209792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1030/2045] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:35.223415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:35.225745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:35.320402 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97620f-95cd-469c-97ba-4798d3acae76 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:35.326489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b7ea9-0081-4412-9c60-3091117507f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:35.327082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b7ea9-0081-4412-9c60-3091117507f2 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:35.327393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b7ea9-0081-4412-9c60-3091117507f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:35.327669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b7ea9-0081-4412-9c60-3091117507f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:35.336976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b7ea9-0081-4412-9c60-3091117507f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:35.338617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b7ea9-0081-4412-9c60-3091117507f2 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:35.348930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1016/2046] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:39.839440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:39.841255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:39.928954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de7042-686d-44a2-a536-6e52ef82ee4c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:39.934245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:39.935276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:39.935645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:39.935920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] PATH_INFO: `/v3/users/3ddabb4633d44d5daff850dca268099d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:39.937258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ddabb4633d44d5daff850dca268099d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:39.939283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:39.986165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:39.999726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ddabb4633d44d5daff850dca268099d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.015587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ddabb4633d44d5daff850dca268099d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.023410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ddabb4633d44d5daff850dca268099d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.031055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ddabb4633d44d5daff850dca268099d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.031764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ddabb4633d44d5daff850dca268099d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.038245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1031/2047] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:39 2022] DELETE /identity/v3/users/3ddabb4633d44d5daff850dca268099d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:40.042843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:40.043482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:40.043695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:40.065074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:40.066324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:40.133810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1017/2048] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:40.144188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:40.145699 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:40.210818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f53f6-4eb2-4e66-b2b1-e93195650ddc admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:40.216056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:40.216780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:40.217065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:40.217338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:40.228571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:40.230039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:40.241081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1032/2049] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:40.533708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:40.535198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:40.548140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57ec8e-c4a1-4bae-bef3-36850f3f17be None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:40.553075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:40.553967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:40.554339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:40.554583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] PATH_INFO: `/v3/projects/2d9194371a964aa7b7c2d0c59dbfc259` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:40.556537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9194371a964aa7b7c2d0c59dbfc259)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:40.558547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:40.576426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:40.610456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9194371a964aa7b7c2d0c59dbfc259', 'request_id': 'req-5f045d27-c50a-416d-91c9-6b9ca4e45d77'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.616080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9194371a964aa7b7c2d0c59dbfc259', 'request_id': 'req-5f045d27-c50a-416d-91c9-6b9ca4e45d77'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.623222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9194371a964aa7b7c2d0c59dbfc259', 'request_id': 'req-5f045d27-c50a-416d-91c9-6b9ca4e45d77'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.637966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] The token cache is being invalidate because project 2d9194371a964aa7b7c2d0c59dbfc259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:40.638297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.639012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.640704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1018/2050] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:40 2022] DELETE /identity/v3/projects/2d9194371a964aa7b7c2d0c59dbfc259 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:40.652164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:40.656229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799095b-7256-4fd5-8122-2994617cb9f3 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:40.723646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:40.724546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:40.724833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:40.725111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] PATH_INFO: `/v3/users/bb43958cfca644f8abf941924ff8ad7d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:40.726383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb43958cfca644f8abf941924ff8ad7d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:40.728731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:40.776778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb43958cfca644f8abf941924ff8ad7d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.783924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:40.790455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb43958cfca644f8abf941924ff8ad7d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.798308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb43958cfca644f8abf941924ff8ad7d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.805396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb43958cfca644f8abf941924ff8ad7d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.806050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac332827-32f5-4535-b1d4-b0759dbd6b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb43958cfca644f8abf941924ff8ad7d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:40.812083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1033/2051] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:40 2022] DELETE /identity/v3/users/bb43958cfca644f8abf941924ff8ad7d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:41.168645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.170351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:41.185371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f045d27-c50a-416d-91c9-6b9ca4e45d77 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.260732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.261528 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.261811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.262072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] PATH_INFO: `/v3/projects/c8aed8644e3d4956b56b189336cd2d6f` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.263309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8aed8644e3d4956b56b189336cd2d6f)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.265266 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.316682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8aed8644e3d4956b56b189336cd2d6f', 'request_id': 'req-e0f63045-54f9-42ff-a998-a29a9ca6b605'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:41.323594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8aed8644e3d4956b56b189336cd2d6f', 'request_id': 'req-e0f63045-54f9-42ff-a998-a29a9ca6b605'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:41.330510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8aed8644e3d4956b56b189336cd2d6f', 'request_id': 'req-e0f63045-54f9-42ff-a998-a29a9ca6b605'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:41.346320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] The token cache is being invalidate because project c8aed8644e3d4956b56b189336cd2d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:41.346634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:41.347287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e0f63045-54f9-42ff-a998-a29a9ca6b605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:41.348890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1019/2052] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:41 2022] DELETE /identity/v3/projects/c8aed8644e3d4956b56b189336cd2d6f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:41.357351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.357854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.357854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.379091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:41.380302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:41.443195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1034/2053] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:41.452926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.453252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.453524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.473273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:41.474489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:41.484103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1020/2054] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:41.492058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.503763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb91161-d306-46c7-9ddc-0205c1e91635 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.509284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.510017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.510238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.510511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.511346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:41.511599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.513235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.526682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1035/2055] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:41.531345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.541638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a06e9-ab90-4614-8e9d-609862639d30 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.546441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.547119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.547399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.547667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.548359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.550272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.578290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1021/2056] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:41.583288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.595799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853679e-a6b8-45b2-b3a7-b1700825e1fd admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.656420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.657109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.657391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.657651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.658355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.660351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.682561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1036/2057] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:41 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:41.686964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.694140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:41.698776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31b5a-19d6-4089-a7b2-8fe28f918578 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.709461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.711153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.711504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.711809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.712810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.714937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.727411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1022/2058] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:41.732442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.743193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65a190-1bf3-4d25-bb90-42d4d3611935 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.748640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efe057-4ace-460b-a9ff-f5437c2e84ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.749587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe057-4ace-460b-a9ff-f5437c2e84ba admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.750032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe057-4ace-460b-a9ff-f5437c2e84ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.750533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe057-4ace-460b-a9ff-f5437c2e84ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.751674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efe057-4ace-460b-a9ff-f5437c2e84ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.754008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efe057-4ace-460b-a9ff-f5437c2e84ba admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.768227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1037/2059] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:41.772604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:41.783384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b952f96-5a3b-49dd-8f8f-8530e4fe3f23 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:41.788715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:41.789676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.789956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.790232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] PATH_INFO: `/v3/projects/4773dc0fca68414c9e07daf19e47efc8/users/d6d60d6276ae427d8a877e671e2343bb/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.806131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4773dc0fca68414c9e07daf19e47efc8, user_id=d6d60d6276ae427d8a877e671e2343bb, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:41.808926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:41.824101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1023/2060] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:41 2022] PUT /identity/v3/projects/4773dc0fca68414c9e07daf19e47efc8/users/d6d60d6276ae427d8a877e671e2343bb/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:41.837318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:41.837609 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:41.837930 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:41.870271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] MFA Rules not processed for user `d6d60d6276ae427d8a877e671e2343bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:41.871892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:41.904170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:41.942383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1038/2061] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:41.947618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.017042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc4dfd-b171-4b10-bbfe-c2911b5463fa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.022783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.023398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.023968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.024375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.025103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.027197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.072118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1024/2062] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:41 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:42.076424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.088822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45761027-2bf7-48ab-8faa-82e1b77bc700 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.165795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.166497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.166804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.167083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.167782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.169716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.192890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1039/2063] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:42 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:42.197278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.207988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e3d1-5c58-4b34-85b6-a45e7b337030 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.220634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.221328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.221611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.221886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.222695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.224388 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.237506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1025/2064] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:42.241652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.253490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656149aa-11f7-4b16-887d-4ffa1127c1ca admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.259379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.260294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.260542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.260830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] PATH_INFO: `/v3/projects/654eff9dd5874a5e9d7c6d59a5f4b5b8/users/e0befe3198b241f38c916725603699ac/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.278488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654eff9dd5874a5e9d7c6d59a5f4b5b8, user_id=e0befe3198b241f38c916725603699ac, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.281139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.296756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1040/2065] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:42 2022] PUT /identity/v3/projects/654eff9dd5874a5e9d7c6d59a5f4b5b8/users/e0befe3198b241f38c916725603699ac/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:42.300846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.313173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dab43-a5e0-4447-967b-8f020fae2aab admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.399944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da523fc-fba7-44ff-b0ec-8fdf7cda7307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.400766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da523fc-fba7-44ff-b0ec-8fdf7cda7307 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.401143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da523fc-fba7-44ff-b0ec-8fdf7cda7307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.401457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da523fc-fba7-44ff-b0ec-8fdf7cda7307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.402229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da523fc-fba7-44ff-b0ec-8fdf7cda7307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.403938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da523fc-fba7-44ff-b0ec-8fdf7cda7307 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.415855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1026/2066] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:42.420342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.431893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a668-392c-4127-b4dd-9fffc190d627 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.437109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.438079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.438364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.438643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] PATH_INFO: `/v3/projects/654eff9dd5874a5e9d7c6d59a5f4b5b8/users/e0befe3198b241f38c916725603699ac/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.440777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654eff9dd5874a5e9d7c6d59a5f4b5b8, user_id=e0befe3198b241f38c916725603699ac, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.443760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.455201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1041/2067] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:42 2022] PUT /identity/v3/projects/654eff9dd5874a5e9d7c6d59a5f4b5b8/users/e0befe3198b241f38c916725603699ac/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:42.467667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.468077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.468460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.497489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] MFA Rules not processed for user `e0befe3198b241f38c916725603699ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:42.498748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:42.532765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:42.575390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1027/2068] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:42.587268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:42.589947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:42.663154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35bfa-d7a7-4124-807e-cefe966ea775 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:42.667423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:42.668097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:42.668380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:42.668654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:42.679048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:42.680536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:42.690588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1042/2069] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:43.056448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:43.058376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:43.120125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:43.135013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aed8f-6c80-4f7a-83fc-0843584336b8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:43.139851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:43.140814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:43.141296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:43.141636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] PATH_INFO: `/v3/users/d8b1c1ac93024141a573d582f710d9d9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:43.142958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b1c1ac93024141a573d582f710d9d9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:43.144968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:43.148471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:43.191787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad7de-3e8d-42b4-9b23-52745990d32e service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:43.196475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:43.197300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b1c1ac93024141a573d582f710d9d9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.197758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:43.197758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:43.197942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] PATH_INFO: `/v3/users/d6d60d6276ae427d8a877e671e2343bb` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:43.199196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d60d6276ae427d8a877e671e2343bb)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:43.201146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:43.209782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b1c1ac93024141a573d582f710d9d9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.219396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b1c1ac93024141a573d582f710d9d9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.227378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b1c1ac93024141a573d582f710d9d9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.228276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-efa2e12a-7963-4b07-bf18-c8998a8a1d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b1c1ac93024141a573d582f710d9d9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.234921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1028/2070] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:43 2022] DELETE /identity/v3/users/d8b1c1ac93024141a573d582f710d9d9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:43.240166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:43.240477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:43.240796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:43.263663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:43.264906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:43.275539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d60d6276ae427d8a877e671e2343bb'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.288583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d60d6276ae427d8a877e671e2343bb'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.298605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d60d6276ae427d8a877e671e2343bb'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.308672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d60d6276ae427d8a877e671e2343bb'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.309399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-abea6bc2-2477-4aaa-b4f6-a617ebde27e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d60d6276ae427d8a877e671e2343bb'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:43.316146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1043/2071] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:43 2022] DELETE /identity/v3/users/d6d60d6276ae427d8a877e671e2343bb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:43.319300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:43.320281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:43.320735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:43.321039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:43.339045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:43.340281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:43.340413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1029/2072] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:43.350653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:43.352652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:43.411288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1044/2073] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:43.418184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a33850-d75f-4fc9-a76a-c495d93b963c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:43.422295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:43.481182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:43.482185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:43.482383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:43.482729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:43.497785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:43.499378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:43.509728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:43.510574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1030/2074] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:17:43.522735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1933d1b-f1c0-421e-b50c-961908c12f7e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:43.528742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:43.529722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:43.530061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:43.530449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:43.542386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:43.543947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:43.555333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1045/2075] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:44.001495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.003101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.026313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ff8c-a3e9-46d6-863e-2375f5a939e9 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.033258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.035134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.035420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.035701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] PATH_INFO: `/v3/projects/2296bbe541c949c8a5383a7ac8f98c4b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.037178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2296bbe541c949c8a5383a7ac8f98c4b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.039550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.057870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.059971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.089443 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cffbe1-7bc7-4828-bdde-565430ad0dab service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.097368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.104921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.105208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.105480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] PATH_INFO: `/v3/projects/4773dc0fca68414c9e07daf19e47efc8` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.106925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4773dc0fca68414c9e07daf19e47efc8)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.109085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.113369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2296bbe541c949c8a5383a7ac8f98c4b', 'request_id': 'req-bee4ea4c-0584-47b6-b951-10f9f4ed4065'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.123092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2296bbe541c949c8a5383a7ac8f98c4b', 'request_id': 'req-bee4ea4c-0584-47b6-b951-10f9f4ed4065'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.132225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2296bbe541c949c8a5383a7ac8f98c4b', 'request_id': 'req-bee4ea4c-0584-47b6-b951-10f9f4ed4065'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.149369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] The token cache is being invalidate because project 2296bbe541c949c8a5383a7ac8f98c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:44.149663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.150329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee4ea4c-0584-47b6-b951-10f9f4ed4065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.152050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1031/2076] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:44 2022] DELETE /identity/v3/projects/2296bbe541c949c8a5383a7ac8f98c4b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:44.156713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e639e4-e32d-488a-8a22-80df10f298a0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.157007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e639e4-e32d-488a-8a22-80df10f298a0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.157600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e639e4-e32d-488a-8a22-80df10f298a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.167756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4773dc0fca68414c9e07daf19e47efc8', 'request_id': 'req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.174205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4773dc0fca68414c9e07daf19e47efc8', 'request_id': 'req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.176651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c3e639e4-e32d-488a-8a22-80df10f298a0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:44.177958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c3e639e4-e32d-488a-8a22-80df10f298a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.182479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4773dc0fca68414c9e07daf19e47efc8', 'request_id': 'req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.194012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] The token cache is being invalidate because project 4773dc0fca68414c9e07daf19e47efc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:44.194291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.194974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.196580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1046/2077] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:44 2022] DELETE /identity/v3/projects/4773dc0fca68414c9e07daf19e47efc8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:44.200680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.201233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:44.202857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.249218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1032/2078] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:44.257722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de97e-69be-484e-981f-95524897a6d6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.258063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de97e-69be-484e-981f-95524897a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.258350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de97e-69be-484e-981f-95524897a6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.273701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac913cc4-bc08-4a0b-8551-5f0a17efb3dc admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.278195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-166de97e-69be-484e-981f-95524897a6d6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:44.279661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-166de97e-69be-484e-981f-95524897a6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.283424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.284441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.284731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.285008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] PATH_INFO: `/v3/users/e0befe3198b241f38c916725603699ac` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.286318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0befe3198b241f38c916725603699ac)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.288150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1033/2079] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:44.288337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.292985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166de97e-69be-484e-981f-95524897a6d6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.303372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166de97e-69be-484e-981f-95524897a6d6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.309324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.310236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.310624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.311040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.311926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:44.312419 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.314172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.331813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1034/2080] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:17:44.334748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.346635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84a21a-2f28-4eab-9850-0622e7a430bd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.352361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.353288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.353814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.354184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.355149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.356549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0befe3198b241f38c916725603699ac'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.357678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.376222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0befe3198b241f38c916725603699ac'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.385106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0befe3198b241f38c916725603699ac'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.386605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:44.391004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1035/2081] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:44 2022] POST /identity/v3/projects => generated 361 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:44.395178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.399347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0befe3198b241f38c916725603699ac'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.400204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0befe3198b241f38c916725603699ac'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:44.407922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1047/2082] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:44 2022] DELETE /identity/v3/users/e0befe3198b241f38c916725603699ac => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:44.469490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe25453-92ca-4153-90fb-75d21bd4d9c7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.533360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.534041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.534320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.534610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.535356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.537297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.559221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1036/2083] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:44 2022] POST /identity/v3/users => generated 281 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:44.563539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.565067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.582385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19b758-bc95-4472-8476-2c1b048d15a7 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.587569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.588328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.588555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.588833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.589576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.591636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.605920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1048/2084] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:44.607252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.618206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f0c3b-bbad-4154-b014-8424bc43cfc2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.624431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0309cfe-5125-432b-9abd-3ff48aed2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.625324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0309cfe-5125-432b-9abd-3ff48aed2544 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.625623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0309cfe-5125-432b-9abd-3ff48aed2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.625909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0309cfe-5125-432b-9abd-3ff48aed2544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.626902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0309cfe-5125-432b-9abd-3ff48aed2544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.628570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0309cfe-5125-432b-9abd-3ff48aed2544 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.641268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1037/2085] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:44.645949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.658487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aaf0c-d1aa-4bc6-b6be-050661d92a2f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.665354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.666440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.667035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.667314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] PATH_INFO: `/v3/projects/cb2141f0fe53497697d44eaca1707a64/users/032ee6db2d9a40d79b232f445da80bf7/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.685883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2141f0fe53497697d44eaca1707a64, user_id=032ee6db2d9a40d79b232f445da80bf7, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.688795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.718796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1049/2086] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:44 2022] PUT /identity/v3/projects/cb2141f0fe53497697d44eaca1707a64/users/032ee6db2d9a40d79b232f445da80bf7/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:44.720765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.720995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.721274 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.748002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:44.759045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] MFA Rules not processed for user `032ee6db2d9a40d79b232f445da80bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:44.760831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.764900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.766823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:44.890839 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1038/2087] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:44.901611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff212c-4342-46e0-824d-c37941f9bc54 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:44.908281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:44.909165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:44.909716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:44.909988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] PATH_INFO: `/v3/projects/654eff9dd5874a5e9d7c6d59a5f4b5b8` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:44.911538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654eff9dd5874a5e9d7c6d59a5f4b5b8)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:44.913469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:44.953279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:44.955487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.000559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654eff9dd5874a5e9d7c6d59a5f4b5b8', 'request_id': 'req-4141a1af-edaa-417a-a77e-f45bb03b1eb0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.011324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654eff9dd5874a5e9d7c6d59a5f4b5b8', 'request_id': 'req-4141a1af-edaa-417a-a77e-f45bb03b1eb0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.022999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654eff9dd5874a5e9d7c6d59a5f4b5b8', 'request_id': 'req-4141a1af-edaa-417a-a77e-f45bb03b1eb0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.036779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] The token cache is being invalidate because project 654eff9dd5874a5e9d7c6d59a5f4b5b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:45.037383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.038285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4141a1af-edaa-417a-a77e-f45bb03b1eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.040926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1050/2088] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:44 2022] DELETE /identity/v3/projects/654eff9dd5874a5e9d7c6d59a5f4b5b8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:45.045898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ab70-b072-4699-9ab6-a37032b942ee None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.046188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ab70-b072-4699-9ab6-a37032b942ee None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.046465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ab70-b072-4699-9ab6-a37032b942ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.047814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1296-4fa2-4959-9489-8796d743e389 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.067641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-02a4ab70-b072-4699-9ab6-a37032b942ee None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:45.069119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-02a4ab70-b072-4699-9ab6-a37032b942ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.119501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.120208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.120690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.121056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.126634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.133764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:45.167310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1051/2089] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:45.176893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.177197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.177630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.198146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:45.199865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.210398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1052/2090] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:45.215076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.221487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.223756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.235985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743bdd7-90ea-4c6f-b17c-88b0301876ed None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.236784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1039/2091] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:45.242259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.243135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.243479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.243830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.244712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:45.246809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.247197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.252698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.254684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.269028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1053/2092] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:45.279596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.279829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba56601-87d4-4f75-8de1-61b4cc863d50 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.285073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595f615-3edd-4ce5-8bd1-34f274b52d76 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.291744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.292616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.292772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.293056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.293350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.293724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.293958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.294235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] PATH_INFO: `/v3/users/032ee6db2d9a40d79b232f445da80bf7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.294383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.295798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=032ee6db2d9a40d79b232f445da80bf7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.296520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.297748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.304446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:45.339563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1054/2093] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/projects => generated 357 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:45.344994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.363216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032ee6db2d9a40d79b232f445da80bf7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.382265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032ee6db2d9a40d79b232f445da80bf7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.392511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032ee6db2d9a40d79b232f445da80bf7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.406503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032ee6db2d9a40d79b232f445da80bf7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.407470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f283c4ef-cd9a-466f-9613-38a0301389ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032ee6db2d9a40d79b232f445da80bf7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:45.418340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1040/2094] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:45 2022] DELETE /identity/v3/users/032ee6db2d9a40d79b232f445da80bf7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:45.423990 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60273481-efe6-4b79-b74f-31590a956b24 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.424782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.425354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.425673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.447145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:45.448492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.492119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.493224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.493629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.493902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.494614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.496700 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.519824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1055/2095] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/users => generated 279 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:45.523846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.525784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.536379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1041/2096] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:45.539440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c7c58-1711-420d-bbb7-b9dd3ff08c47 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.544282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.544966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.545250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.545531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.546270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.547539 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.548058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.549013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.558983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1056/2097] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:45.562971 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.573856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036eb6a-4eb7-4e29-b5a3-e6db421718e4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.578950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.579625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.579907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.580186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.580927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.582547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.595138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1057/2098] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:45.599406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.609802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0668-09a7-458d-be9f-351d5cfb50a9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.614846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad21cc-f2aa-4db5-ba01-aedd8a6d08bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.615676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad21cc-f2aa-4db5-ba01-aedd8a6d08bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.615949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad21cc-f2aa-4db5-ba01-aedd8a6d08bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.616214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad21cc-f2aa-4db5-ba01-aedd8a6d08bf admin admin] PATH_INFO: `/v3/projects/7e2ef17ac5be4eae9f94e6028983bfb6/users/5f879e50ee1c429b951599ab39dd24d0/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.628534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66295a5d-14c4-449c-ba8b-334e9b4c43d8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.631140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad21cc-f2aa-4db5-ba01-aedd8a6d08bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2ef17ac5be4eae9f94e6028983bfb6, user_id=5f879e50ee1c429b951599ab39dd24d0, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.633898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad21cc-f2aa-4db5-ba01-aedd8a6d08bf admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.634481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.635592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.636146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.636253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:45.636629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.650424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1058/2099] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:45 2022] PUT /identity/v3/projects/7e2ef17ac5be4eae9f94e6028983bfb6/users/5f879e50ee1c429b951599ab39dd24d0/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:45.650787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.652259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.660815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.661099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.661378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.662947 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1042/2100] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:45.700645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] MFA Rules not processed for user `5f879e50ee1c429b951599ab39dd24d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:45.702525 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:45.780527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1059/2101] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:45.785290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.796366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804803d-dc20-49fc-8210-bc3e3815b6de service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.829906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:45.878551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:45.879447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:45.879898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:45.880305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:45.881331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:45.884033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:45.917813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1043/2102] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/projects => generated 355 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:45.921819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:45.935184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc6bf5-5095-437c-b4d2-2aa454034157 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:45.999310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.001099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.006819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.007614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.007798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.008304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.009220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.012425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.016713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da75ca7-86d4-47fb-86ce-f289563a274e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.022119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.023099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.023782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.023782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] PATH_INFO: `/v3/projects/cb2141f0fe53497697d44eaca1707a64` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.025722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2141f0fe53497697d44eaca1707a64)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.027759 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.059819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1060/2103] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:45 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:46.060887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.074565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35a5e-41dc-4578-9988-0b1abec84691 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.086244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.087135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.087507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.087851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.088715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.091520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.104060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1061/2104] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:46.108131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.114087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2141f0fe53497697d44eaca1707a64', 'request_id': 'req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.119119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddf84d-d468-4e55-a5d6-5af0e0d10224 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.122811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2141f0fe53497697d44eaca1707a64', 'request_id': 'req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.144179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2141f0fe53497697d44eaca1707a64', 'request_id': 'req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.164416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] The token cache is being invalidate because project cb2141f0fe53497697d44eaca1707a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:46.164828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.166534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5b07f5e2-e5c6-48ae-84c1-e12e5e61a779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.171836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1044/2105] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:45 2022] DELETE /identity/v3/projects/cb2141f0fe53497697d44eaca1707a64 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:46.174558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.174961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.175417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.185763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2880-0527-4dda-8c4d-73bcea9d029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.194332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2880-0527-4dda-8c4d-73bcea9d029b admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.194947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2880-0527-4dda-8c4d-73bcea9d029b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.195245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2880-0527-4dda-8c4d-73bcea9d029b admin admin] PATH_INFO: `/v3/projects/a6422b4f904b48d09b53fa960c54140a/users/e6766871602d45b4ac66868b2dbc678f/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.199120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:46.205734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:46.207971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.214376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d2880-0527-4dda-8c4d-73bcea9d029b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6422b4f904b48d09b53fa960c54140a, user_id=e6766871602d45b4ac66868b2dbc678f, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.217600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d2880-0527-4dda-8c4d-73bcea9d029b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.222854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1045/2106] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:46.222854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.233260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.248905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7e19d-6fa2-465e-9d75-3ca33ef8e87d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.252566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1062/2107] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:46 2022] PUT /identity/v3/projects/a6422b4f904b48d09b53fa960c54140a/users/e6766871602d45b4ac66868b2dbc678f/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:46.258139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.258594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.258872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.282228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:46.283876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.324064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.324968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.325249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.325521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] PATH_INFO: `/v3/users/50944673799940368982b7c196eb65b3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.326793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=50944673799940368982b7c196eb65b3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.329081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.368929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1063/2108] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:46.371756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.372876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:46.373291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.389592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7a4-d7ae-41c4-bb0c-ac8d7bd76721 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.389820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50944673799940368982b7c196eb65b3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.394586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.395312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.395723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.395898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.396614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.398251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.405662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50944673799940368982b7c196eb65b3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.412837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1064/2109] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:46.412837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.413537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.421823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50944673799940368982b7c196eb65b3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.429365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81cd4-3930-49fa-a24e-10de0ed01acf admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.431925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50944673799940368982b7c196eb65b3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.432657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50944673799940368982b7c196eb65b3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.435506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.436528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.436818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.437100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] PATH_INFO: `/v3/users/b4f0a8c23ede41659a3963037e738219` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.438377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f0a8c23ede41659a3963037e738219)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.440543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1046/2110] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:46 2022] DELETE /identity/v3/users/50944673799940368982b7c196eb65b3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:46.441449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.443012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.443427 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.521671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f0a8c23ede41659a3963037e738219'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.537938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f0a8c23ede41659a3963037e738219'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.547473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f0a8c23ede41659a3963037e738219'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.555948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f0a8c23ede41659a3963037e738219'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.556751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f0a8c23ede41659a3963037e738219'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:46.558870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59290877-ccaf-4cbc-be84-c54b1b59fa4a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.564242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1065/2111] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:46 2022] DELETE /identity/v3/users/b4f0a8c23ede41659a3963037e738219 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:46.567452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.567452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.567452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.567452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.567803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:46.567803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.569130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.569637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d82349-9e8c-4e35-adb5-dcd63fa3c02e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.571172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.582978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1047/2112] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:17:46.589154 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59f578-df27-4505-adc2-2d2e860e1fb8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.589558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59f578-df27-4505-adc2-2d2e860e1fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.589869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59f578-df27-4505-adc2-2d2e860e1fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.609307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2a59f578-df27-4505-adc2-2d2e860e1fb8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:46.610550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2a59f578-df27-4505-adc2-2d2e860e1fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.627844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:46.650429 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03846a-99ba-407e-9cb0-7ec8670007a0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.666144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.667908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.668384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.668658 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] PATH_INFO: `/v3/projects/a6422b4f904b48d09b53fa960c54140a/users/e6766871602d45b4ac66868b2dbc678f/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.671706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6422b4f904b48d09b53fa960c54140a, user_id=e6766871602d45b4ac66868b2dbc678f, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.677050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.683732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1048/2113] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:46.688583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.688948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.689827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.698790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1066/2114] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:46 2022] PUT /identity/v3/projects/a6422b4f904b48d09b53fa960c54140a/users/e6766871602d45b4ac66868b2dbc678f/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:46.699748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.709744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.713393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:46.714675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.761005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:46.793669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1049/2115] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:46.794895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.796641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.819440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e229f-78d6-4428-b962-a15ac57c684d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.825277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49153f7f-3639-4d1b-9984-626954a8f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.826047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49153f7f-3639-4d1b-9984-626954a8f38b admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.826406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49153f7f-3639-4d1b-9984-626954a8f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.826777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49153f7f-3639-4d1b-9984-626954a8f38b admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.827571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49153f7f-3639-4d1b-9984-626954a8f38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:46.830018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49153f7f-3639-4d1b-9984-626954a8f38b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:46.868368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1067/2116] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:46 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:46.868368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.868368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.868368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.868762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48283c-e010-4c30-9f7c-abb0c39fddea None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.897638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] MFA Rules not processed for user `e6766871602d45b4ac66868b2dbc678f`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:46.899048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:46.931595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.932285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.932571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.932847 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:46.973970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1068/2117] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:46.978619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:46.991533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a21eb5-70a7-4cab-a924-ac6eef06ba70 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:46.995654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:46.996315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:46.996601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:46.996881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.002211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.003706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.006652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.008317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.013939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1050/2118] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:47.014865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.018620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1069/2119] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:47.019703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.021269 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.025314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded60461-cb92-4ef9-bc6c-2a103628f024 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.030303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.031021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.031366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.031646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.032360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.034306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.059474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1051/2120] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:47 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:47.060875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.072573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:47.073838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfc4d8-e586-42d2-8367-3fa2d49e8b57 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.080172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.080938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.081175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.081538 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.082380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.084089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.093303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646e118-7763-4114-a799-6b160ba7f97d service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.096272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1052/2121] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:47.100688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.101012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.101986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.102273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.102551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.114157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eff0d-6579-4610-8616-828db5ca4270 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.114618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.116584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.120727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.121573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.122220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.122220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.123173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.125089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.129234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1070/2122] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:47.146416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1053/2123] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:47.147843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.161417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea93338-f3fd-4c66-97f2-4daa11248611 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.167206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef80965-f852-4a1c-8094-da1c4d6b3eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.168129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80965-f852-4a1c-8094-da1c4d6b3eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.168412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80965-f852-4a1c-8094-da1c4d6b3eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.169263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef80965-f852-4a1c-8094-da1c4d6b3eaf admin admin] PATH_INFO: `/v3/projects/66931d9e9d4849ab9c78bb11624e3177/users/56bc9a55f0f74abca0d8e8606907cffb/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.202782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef80965-f852-4a1c-8094-da1c4d6b3eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=66931d9e9d4849ab9c78bb11624e3177, user_id=56bc9a55f0f74abca0d8e8606907cffb, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.208371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef80965-f852-4a1c-8094-da1c4d6b3eaf admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.216953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.225940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1071/2124] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:47 2022] PUT /identity/v3/projects/66931d9e9d4849ab9c78bb11624e3177/users/56bc9a55f0f74abca0d8e8606907cffb/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:47.228242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9318-6d52-4961-ae14-8cc9aa27cc3d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.234389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:47.239087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.239424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.239655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.269044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] MFA Rules not processed for user `56bc9a55f0f74abca0d8e8606907cffb`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:47.270229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.285327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.285991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.286273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.286549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.288568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.391929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.393576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.397935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1072/2125] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:47.405207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1054/2126] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:47.411050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.412704 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.427907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ea292-baf1-4eaf-b7b1-5e520cdcb034 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.433258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.433809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.434195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.434577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.447806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.450087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.463943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1073/2127] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:47.495622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.497360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.532473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.534370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.626581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918430f-24f8-47b5-8dc2-e77df3b39d5a service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.638430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.639872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.641347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.641784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] PATH_INFO: `/v3/projects/f389b844cee34312a4c6640284c53ef0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.643635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f389b844cee34312a4c6640284c53ef0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.647400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.648365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bdf1a-08e3-4bfe-98b0-6dd5bc6d58f6 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.653280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:47.654178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.654457 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.654761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] PATH_INFO: `/v3/projects/3d4512d231694fa187ada8905222baf7` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.656164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4512d231694fa187ada8905222baf7)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:47.658105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:47.723065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:47.729131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f389b844cee34312a4c6640284c53ef0', 'request_id': 'req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.736359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f389b844cee34312a4c6640284c53ef0', 'request_id': 'req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.736732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4512d231694fa187ada8905222baf7', 'request_id': 'req-a6f28659-d819-41de-a75a-d7ad12580467'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.744946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4512d231694fa187ada8905222baf7', 'request_id': 'req-a6f28659-d819-41de-a75a-d7ad12580467'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.745984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f389b844cee34312a4c6640284c53ef0', 'request_id': 'req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.750490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:47.754310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4512d231694fa187ada8905222baf7', 'request_id': 'req-a6f28659-d819-41de-a75a-d7ad12580467'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.765421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] The token cache is being invalidate because project f389b844cee34312a4c6640284c53ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:47.765855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.766914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6d0f95a8-660b-4c91-8bc5-34aedb12ecc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.769067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1055/2128] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:47 2022] DELETE /identity/v3/projects/f389b844cee34312a4c6640284c53ef0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:47.769230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] The token cache is being invalidate because project 3d4512d231694fa187ada8905222baf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:47.769371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.769995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a6f28659-d819-41de-a75a-d7ad12580467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:47.771844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1074/2129] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:47 2022] DELETE /identity/v3/projects/3d4512d231694fa187ada8905222baf7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:47.776483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64a61-90c6-4c47-a0cc-6717bf12b7ef None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.776771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0482c-084d-42ab-beff-4a6e621ae646 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.776875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64a61-90c6-4c47-a0cc-6717bf12b7ef None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.777041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64a61-90c6-4c47-a0cc-6717bf12b7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.777391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0482c-084d-42ab-beff-4a6e621ae646 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.777744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0482c-084d-42ab-beff-4a6e621ae646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.798067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cbb0482c-084d-42ab-beff-4a6e621ae646 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:47.798327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f4c64a61-90c6-4c47-a0cc-6717bf12b7ef None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:47.799407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f4c64a61-90c6-4c47-a0cc-6717bf12b7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.800526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cbb0482c-084d-42ab-beff-4a6e621ae646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.895573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1075/2130] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:47.918874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1056/2131] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:47.923034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.923361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.923660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.924216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:47.924705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:47.925186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:47.963566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:47.965124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.965896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:47.967731 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:47.976829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1076/2132] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:47.980394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1057/2133] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:47.984025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.984369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:47.996738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222e16a-4185-4481-bb08-7d0c1335a87e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:47.997130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde413-f2b8-471f-8f16-9f8c63efb439 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.005218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.005692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.006879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.007380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.007736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.008106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.008445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.008800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.010491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:48.010837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.011772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:17:48.012491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.012939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.015181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.029774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1058/2134] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:17:48.032954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1077/2135] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:17:48.034733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.037513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.047409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b3377-adf2-4c2e-8bf5-471f15543525 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.050450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a45583-2a58-43ad-831f-5c0e26ff9112 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.052495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.053218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.053503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.053787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.054558 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.056678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.057088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.057784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.058068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.058345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.059115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.061137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.092483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1059/2136] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:48 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:48.095507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.101177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1078/2137] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:17:48 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:48.103550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.174497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a8f7-1767-4637-a84a-dffd56785e7e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.215124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26205c-c591-46ed-aa1a-42b66a28be1e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.222026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.223201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.223383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.223764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.224707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.227217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.255085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1079/2138] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:48 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:17:48.259901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.260525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.261411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.261756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.262116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.263203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.266258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.274659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:48.282287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f802b-ffb3-4d5f-a347-cf276da77d6d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.288401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.289174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.289539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.289814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.290651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.292616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.295479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1060/2139] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:17:48 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:17:48.301021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.302906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:48.309170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1080/2140] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:48.313615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6787-5b4b-4ba9-9b10-43aec0c78830 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.319594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.320138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.320503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.320807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.321228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.321706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.323591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.334997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1061/2141] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:48.335417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a959-da0d-45b3-badc-3a1b16ef8b64 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.339140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.341139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.341815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.342109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.342387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.343155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.345014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.349507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfabd97-c61f-4ac3-a32b-bdaa9e6890f3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.354386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.355081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.355364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.355640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.356372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.357995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.358863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1081/2142] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:17:48.361435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.370452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1062/2143] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:17:48.374966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.378387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bade1-0f18-426d-8747-45d6c3d5f004 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.383412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f379e-effb-4f4d-badd-ebb57fbb83ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.384327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f379e-effb-4f4d-badd-ebb57fbb83ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.384596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f379e-effb-4f4d-badd-ebb57fbb83ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.384870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f379e-effb-4f4d-badd-ebb57fbb83ac admin admin] PATH_INFO: `/v3/projects/ce278ea710f64c04b28929b9856e1875/users/e40ef35d4c8c41ba845555c9953c17fd/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.385482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c4976-1f19-426f-951b-e7e7d11dcfe3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.391200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc7530-a440-4a44-a087-33e31b474db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.392084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7530-a440-4a44-a087-33e31b474db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.392556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7530-a440-4a44-a087-33e31b474db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.392840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7530-a440-4a44-a087-33e31b474db1 admin admin] PATH_INFO: `/v3/projects/64f4d8d12085443eb16c2bade44938c6/users/47ce7d48bd2541ada96046818a9ce1d8/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.411923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f379e-effb-4f4d-badd-ebb57fbb83ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce278ea710f64c04b28929b9856e1875, user_id=e40ef35d4c8c41ba845555c9953c17fd, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.415892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f379e-effb-4f4d-badd-ebb57fbb83ac admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.422255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc7530-a440-4a44-a087-33e31b474db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f4d8d12085443eb16c2bade44938c6, user_id=47ce7d48bd2541ada96046818a9ce1d8, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.426219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc7530-a440-4a44-a087-33e31b474db1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.442517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1063/2144] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:48 2022] PUT /identity/v3/projects/64f4d8d12085443eb16c2bade44938c6/users/47ce7d48bd2541ada96046818a9ce1d8/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:17:48.447473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1082/2145] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:17:48 2022] PUT /identity/v3/projects/ce278ea710f64c04b28929b9856e1875/users/e40ef35d4c8c41ba845555c9953c17fd/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:48.453341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.454012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.454305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.462891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.463179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.463454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.485643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] MFA Rules not processed for user `47ce7d48bd2541ada96046818a9ce1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:48.487128 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:48.497261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] MFA Rules not processed for user `e40ef35d4c8c41ba845555c9953c17fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:48.498490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:48.559686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1064/2146] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:48.570967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.572488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:48.573924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1083/2147] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:17:48.586296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:48.589301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:48.633422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacdf4f-8a33-4f22-8f0f-b2b7fa915733 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.637720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.638437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.638738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.639021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.648011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.650059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.652036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc313-6b62-47b9-ae13-51252b7c50d5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:48.656193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97f545-61bf-4f9d-ae22-f1a1ae73e9dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:48.656928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97f545-61bf-4f9d-ae22-f1a1ae73e9dc service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:48.657215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97f545-61bf-4f9d-ae22-f1a1ae73e9dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:48.657493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97f545-61bf-4f9d-ae22-f1a1ae73e9dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:48.667623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97f545-61bf-4f9d-ae22-f1a1ae73e9dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:48.668835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1065/2148] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:17:48.669422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97f545-61bf-4f9d-ae22-f1a1ae73e9dc service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:48.680278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1084/2149] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:17:53.129525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:53.174355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:53.208359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda4879-0ef2-468f-acb9-9e6d26e2f789 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:53.213360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:53.214285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:53.214606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:53.214916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] PATH_INFO: `/v3/users/47ce7d48bd2541ada96046818a9ce1d8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:53.216547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ce7d48bd2541ada96046818a9ce1d8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:53.218507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:53.266565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ce7d48bd2541ada96046818a9ce1d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.279663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ce7d48bd2541ada96046818a9ce1d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.287607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ce7d48bd2541ada96046818a9ce1d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.295205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ce7d48bd2541ada96046818a9ce1d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.295872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ce7d48bd2541ada96046818a9ce1d8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.302028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1066/2150] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:53 2022] DELETE /identity/v3/users/47ce7d48bd2541ada96046818a9ce1d8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:53.307255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:53.307819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:53.307920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:53.327312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:53.328579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:53.386009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:53.400652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1085/2151] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:53.413908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:53.415591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:53.499264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c83d8d-e65b-40b2-9a7b-b323c7aee397 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:53.503146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:53.504202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:53.504487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:53.504914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:53.515920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:53.518390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:53.531150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1067/2152] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:17:53.828969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:53.830612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:53.845179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d0ff7-47eb-4599-bb3f-a11bef2fa162 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:53.850487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:53.851396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:53.851679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:53.851965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] PATH_INFO: `/v3/projects/64f4d8d12085443eb16c2bade44938c6` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:53.854578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f4d8d12085443eb16c2bade44938c6)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:53.856670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:53.908121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f4d8d12085443eb16c2bade44938c6', 'request_id': 'req-71484600-21e3-4109-9987-7fd1d9564c8c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.914370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f4d8d12085443eb16c2bade44938c6', 'request_id': 'req-71484600-21e3-4109-9987-7fd1d9564c8c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.922631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f4d8d12085443eb16c2bade44938c6', 'request_id': 'req-71484600-21e3-4109-9987-7fd1d9564c8c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.934349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] The token cache is being invalidate because project 64f4d8d12085443eb16c2bade44938c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:17:53.934635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.935306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-71484600-21e3-4109-9987-7fd1d9564c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:53.936887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1086/2153] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:17:53 2022] DELETE /identity/v3/projects/64f4d8d12085443eb16c2bade44938c6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:59.517967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:59.519582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:59.533403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f353-8ffc-4d2a-961f-b5807a2af18e service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:59.603082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:17:59.610423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:59.611496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:59.612831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:59.612831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] PATH_INFO: `/v3/users/e40ef35d4c8c41ba845555c9953c17fd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:59.613963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40ef35d4c8c41ba845555c9953c17fd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:59.616475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:59.669133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40ef35d4c8c41ba845555c9953c17fd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:59.682347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40ef35d4c8c41ba845555c9953c17fd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:59.691046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40ef35d4c8c41ba845555c9953c17fd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:59.699265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40ef35d4c8c41ba845555c9953c17fd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:59.700215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40ef35d4c8c41ba845555c9953c17fd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:17:59.707343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1068/2154] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:17:59 2022] DELETE /identity/v3/users/e40ef35d4c8c41ba845555c9953c17fd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:17:59.712489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:59.712998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:59.713280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:59.735690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:17:59.736413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:59.799023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1087/2155] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:17:59.810137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:17:59.811656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:17:59.874615 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896980bf-334e-4166-a99a-080262744b80 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:17:59.878842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a328cb8-9859-4792-9c9d-9d4e05bd1169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:17:59.879500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328cb8-9859-4792-9c9d-9d4e05bd1169 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:17:59.879781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328cb8-9859-4792-9c9d-9d4e05bd1169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:17:59.880053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328cb8-9859-4792-9c9d-9d4e05bd1169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:17:59.891292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a328cb8-9859-4792-9c9d-9d4e05bd1169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:17:59.894517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a328cb8-9859-4792-9c9d-9d4e05bd1169 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:17:59.905458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1069/2156] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:00.266886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.269774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:00.286601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6dc0-01b5-4e46-8104-3aa178d2b91b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.292806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.293719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.294036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.294361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] PATH_INFO: `/v3/projects/ce278ea710f64c04b28929b9856e1875` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.295714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce278ea710f64c04b28929b9856e1875)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.297961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.379484 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce278ea710f64c04b28929b9856e1875', 'request_id': 'req-7217b42b-8991-461e-a80a-84158a9ebc8d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:00.388351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce278ea710f64c04b28929b9856e1875', 'request_id': 'req-7217b42b-8991-461e-a80a-84158a9ebc8d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:00.399403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce278ea710f64c04b28929b9856e1875', 'request_id': 'req-7217b42b-8991-461e-a80a-84158a9ebc8d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:00.413308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] The token cache is being invalidate because project ce278ea710f64c04b28929b9856e1875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:00.413606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:00.414272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7217b42b-8991-461e-a80a-84158a9ebc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:00.423000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1088/2157] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:00 2022] DELETE /identity/v3/projects/ce278ea710f64c04b28929b9856e1875 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:00.423106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.423106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.423106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.445270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:00.446607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:00.531202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1070/2158] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:00.542462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.543229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.543359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.551471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:00.564304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:00.566111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:00.582860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1089/2159] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:00.586441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.598094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed496-73e3-4b13-9699-bbc0507b0723 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.605854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.605854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.605854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.605854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.606229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:00.606229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.608083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.626139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1071/2160] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:00.627458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.637692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a2ed1-aafb-4eae-8883-9d9bb1fe6e72 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.642464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.643179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.643458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.643733 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.644423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.646371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.673050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1090/2161] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:00 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:00.677515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.688091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9aac-5035-47fe-8ffd-f899614c2d4f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.692214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:00.758339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.759070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.759337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.759621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.760318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.762445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.784863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1072/2162] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:00 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:00.789675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.801164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70feb0-6c37-4ad5-b9c4-163424e06b77 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.806774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.807474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.807763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.808042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.808955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.810597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.823200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1091/2163] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:00.827787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.838803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08785262-7b2c-466d-bf9d-618a0435478c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.844275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecf851-5c16-47fa-9c76-77fdbdaece00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.844954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf851-5c16-47fa-9c76-77fdbdaece00 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.845242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf851-5c16-47fa-9c76-77fdbdaece00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.845516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf851-5c16-47fa-9c76-77fdbdaece00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.846290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecf851-5c16-47fa-9c76-77fdbdaece00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.848074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecf851-5c16-47fa-9c76-77fdbdaece00 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.859207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1073/2164] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:00.863620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:00.874935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6a83d-4428-41f7-bd3a-73c19e76104f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:00.880747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:00.881783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.881875 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.882181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] PATH_INFO: `/v3/projects/4aa87f65370249549c165b4dc8b82c7b/users/f7f6b7da5c7d426baf4410ac9008007c/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.899154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa87f65370249549c165b4dc8b82c7b, user_id=f7f6b7da5c7d426baf4410ac9008007c, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:00.901828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:00.916876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1092/2165] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:00 2022] PUT /identity/v3/projects/4aa87f65370249549c165b4dc8b82c7b/users/f7f6b7da5c7d426baf4410ac9008007c/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:00.927536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:00.927829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:00.928104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:00.958749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] MFA Rules not processed for user `f7f6b7da5c7d426baf4410ac9008007c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:00.960351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:01.031414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1074/2166] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:01.036298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.102640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b8aa0-179e-45e5-9dca-bd76754966b9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.108536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.109226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.109506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.109782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.110490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.112478 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.142782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1093/2167] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:01 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:01.147249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.159549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2ad06-42ec-419a-bb1c-4ae06b36e418 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.224468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.225148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.225429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.225703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.226413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.228428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.250025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1075/2168] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:01 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:01.254223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.265572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9386ccf-c21f-4ffb-8dd3-fd53158769b5 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.271068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.271735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.272011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.272279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.273008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.274616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.286424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1094/2169] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:01.291432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.298669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:01.303204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75ed30-7541-4027-b51c-35c184687aee admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.309597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91882312-3710-4632-8e47-0380cb42e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.310282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91882312-3710-4632-8e47-0380cb42e9eb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.310569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91882312-3710-4632-8e47-0380cb42e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.310882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91882312-3710-4632-8e47-0380cb42e9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.311629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91882312-3710-4632-8e47-0380cb42e9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.313260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91882312-3710-4632-8e47-0380cb42e9eb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.324036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1076/2170] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:01.328541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.339289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed1c0-a538-463a-b6fe-1c39c13d4081 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.345037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.345994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.346283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.346588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] PATH_INFO: `/v3/projects/d6c0a4e40e3e4e1f8d962e848e11d3c4/users/34c55d67a064402395ed7eba865c96b4/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.364352 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c0a4e40e3e4e1f8d962e848e11d3c4, user_id=34c55d67a064402395ed7eba865c96b4, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.367109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.381978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1095/2171] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:01 2022] PUT /identity/v3/projects/d6c0a4e40e3e4e1f8d962e848e11d3c4/users/34c55d67a064402395ed7eba865c96b4/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:01.392924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.393214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.393543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.431882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] MFA Rules not processed for user `34c55d67a064402395ed7eba865c96b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:01.433330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:01.521764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1077/2172] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:01.528521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.580893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:01.595304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c6d02-8f2b-45d4-8304-d0298f50099f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.600088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.600755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.601045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.601311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.602006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.603953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.634123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1096/2173] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:01 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:01.638329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.654912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd99a2-c707-4be8-a5ad-6a0191d7cca5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.725050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.725803 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.726149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.726494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.727302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.729663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.755271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1078/2174] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:01 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:01.759641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.770678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f516c-343c-4660-ad89-8c37bdfa2bd2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.775811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.776578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.776863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.777195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.777948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.779758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.792063 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1097/2175] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:01.796769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.807162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2df8e-74a4-4793-a83f-267b39f749d5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.811890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.812775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.813055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.813328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] PATH_INFO: `/v3/projects/86bf32696b9942ca845d59170e24e09b/users/386f27696f3b4e2fb01c54228f273fd0/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.828317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bf32696b9942ca845d59170e24e09b, user_id=386f27696f3b4e2fb01c54228f273fd0, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.831006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.845779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1079/2176] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:01 2022] PUT /identity/v3/projects/86bf32696b9942ca845d59170e24e09b/users/386f27696f3b4e2fb01c54228f273fd0/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:01.849938 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.860694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c5708-c5ab-47cf-b216-6903d2f59e16 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.926764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589b42-3918-490f-86c9-90ec027dd845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.927811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589b42-3918-490f-86c9-90ec027dd845 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.928240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589b42-3918-490f-86c9-90ec027dd845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.928838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589b42-3918-490f-86c9-90ec027dd845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.930023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8589b42-3918-490f-86c9-90ec027dd845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.932519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8589b42-3918-490f-86c9-90ec027dd845 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.943786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1098/2177] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:01.948123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:01.959901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57ea4f-fdf7-4cf3-b662-4316d9c04461 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:01.966112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:01.967046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:01.967327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:01.967604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] PATH_INFO: `/v3/projects/86bf32696b9942ca845d59170e24e09b/users/386f27696f3b4e2fb01c54228f273fd0/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:01.969762 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bf32696b9942ca845d59170e24e09b, user_id=386f27696f3b4e2fb01c54228f273fd0, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:01.972486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:01.989518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1080/2178] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:01 2022] PUT /identity/v3/projects/86bf32696b9942ca845d59170e24e09b/users/386f27696f3b4e2fb01c54228f273fd0/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:02.002636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:02.002910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:02.003226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:02.031479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] MFA Rules not processed for user `386f27696f3b4e2fb01c54228f273fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:02.032709 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:02.104898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1099/2179] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:02.117141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:02.120166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:02.165418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:02.191163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619c495-9c7d-445c-9818-c0a2367cca40 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:02.195486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:02.196150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:02.196432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:02.196689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:02.207273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:02.208741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:02.218948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1081/2180] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:02.260323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:02.261987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:02.274529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5901b9-7391-4490-a3f6-f830d1d71f6f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:02.278813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:02.279472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:02.279756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:02.280030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:02.328941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:02.344814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:02.346266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:02.356243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1100/2181] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:18:05.492873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:05.576473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354c65-20f0-414a-be30-76a995593c35 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:05.580998 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:05.582077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:05.582360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:05.582636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] PATH_INFO: `/v3/users/f7f6b7da5c7d426baf4410ac9008007c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:05.583929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f6b7da5c7d426baf4410ac9008007c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:05.586041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:05.659912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:05.661947 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:05.672534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f6b7da5c7d426baf4410ac9008007c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.689107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f6b7da5c7d426baf4410ac9008007c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.692855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2d812-4b41-4555-8d26-f93fabd2b20d service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:05.701197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:05.702336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:05.702877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:05.703327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] PATH_INFO: `/v3/users/5f879e50ee1c429b951599ab39dd24d0` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:05.704942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f879e50ee1c429b951599ab39dd24d0)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:05.708180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:05.710375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f6b7da5c7d426baf4410ac9008007c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.723463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f6b7da5c7d426baf4410ac9008007c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.724246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0b903e9a-421b-4266-bde2-1696c753cc81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f6b7da5c7d426baf4410ac9008007c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.731438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1082/2182] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:05 2022] DELETE /identity/v3/users/f7f6b7da5c7d426baf4410ac9008007c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:05.735891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:05.736297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:05.736636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:05.762146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:05.763023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:05.776671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f879e50ee1c429b951599ab39dd24d0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.792083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f879e50ee1c429b951599ab39dd24d0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.802319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f879e50ee1c429b951599ab39dd24d0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.812439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f879e50ee1c429b951599ab39dd24d0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.813203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9bb9b94e-5f0b-4e69-9dff-104a0e90c218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f879e50ee1c429b951599ab39dd24d0'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:05.820222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1101/2183] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:05 2022] DELETE /identity/v3/users/5f879e50ee1c429b951599ab39dd24d0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:05.825251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:05.825540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:05.825822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:05.831965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:05.851781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:05.853129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:05.857180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1083/2184] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:05.858592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:05.859610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:05.933514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e23c6e-3f7f-46fc-bfc1-d8b030cf2268 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:05.936402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1102/2185] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:05.946255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:06.003099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:06.003748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:06.004027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:06.004294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:06.014424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:06.015919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:06.020828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b1be6-88af-495c-b27e-35c15ca661a8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:06.024612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:06.025278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:06.025562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:06.025859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:06.025993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1084/2186] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:06.036415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:06.037903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:06.043529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:06.049282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1103/2187] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:06.521513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:06.530961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:06.545305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:06.546877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79367b3d-e5d3-4080-8715-baa056ce96c9 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:06.547002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:06.565864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd1131-ffe0-4c67-9435-c56869839c00 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:06.569209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:06.571002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:06.571296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:06.571582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:06.571885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] PATH_INFO: `/v3/projects/4aa87f65370249549c165b4dc8b82c7b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:06.572526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:06.572908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:06.573325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] PATH_INFO: `/v3/projects/7e2ef17ac5be4eae9f94e6028983bfb6` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:06.575159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2ef17ac5be4eae9f94e6028983bfb6)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:06.578643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:06.579348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa87f65370249549c165b4dc8b82c7b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:06.584102 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:06.664093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa87f65370249549c165b4dc8b82c7b', 'request_id': 'req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.671101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2ef17ac5be4eae9f94e6028983bfb6', 'request_id': 'req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.674239 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa87f65370249549c165b4dc8b82c7b', 'request_id': 'req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.681398 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa87f65370249549c165b4dc8b82c7b', 'request_id': 'req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.682492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2ef17ac5be4eae9f94e6028983bfb6', 'request_id': 'req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.704043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2ef17ac5be4eae9f94e6028983bfb6', 'request_id': 'req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.712159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] The token cache is being invalidate because project 4aa87f65370249549c165b4dc8b82c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:06.712454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.713119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.719810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1085/2188] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:06 2022] DELETE /identity/v3/projects/4aa87f65370249549c165b4dc8b82c7b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:06.720930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:06.722382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:06.723053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] The token cache is being invalidate because project 7e2ef17ac5be4eae9f94e6028983bfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:06.723418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.727444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.727444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1104/2189] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:06 2022] DELETE /identity/v3/projects/7e2ef17ac5be4eae9f94e6028983bfb6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:06.730424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:06.732036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:06.780004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:06.802346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d62c2-dccc-43a6-9a4a-8c7eee69af58 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:06.819194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb71aa-c5f6-4336-acfe-2a4d55179d68 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:06.824536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:06.825499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:06.825808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:06.826103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] PATH_INFO: `/v3/users/e6766871602d45b4ac66868b2dbc678f` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:06.827390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6766871602d45b4ac66868b2dbc678f)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:06.829359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:06.867405 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:06.868409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:06.868708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:06.868970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] PATH_INFO: `/v3/users/34c55d67a064402395ed7eba865c96b4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:06.870249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c55d67a064402395ed7eba865c96b4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:06.872456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:06.889759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6766871602d45b4ac66868b2dbc678f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.904558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6766871602d45b4ac66868b2dbc678f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.912437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6766871602d45b4ac66868b2dbc678f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.922815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6766871602d45b4ac66868b2dbc678f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.924276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6766871602d45b4ac66868b2dbc678f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.925003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c55d67a064402395ed7eba865c96b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.932503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1105/2190] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:06 2022] DELETE /identity/v3/users/e6766871602d45b4ac66868b2dbc678f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:06.950991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c55d67a064402395ed7eba865c96b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.960752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c55d67a064402395ed7eba865c96b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.971531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c55d67a064402395ed7eba865c96b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.972800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c55d67a064402395ed7eba865c96b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:06.980238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1086/2191] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:06 2022] DELETE /identity/v3/users/34c55d67a064402395ed7eba865c96b4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:07.276161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:07.277818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.284208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:07.332617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:07.334450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.389056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239a9c-e624-42a6-89c7-a7b536f7c509 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:07.412926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a560b45-18f0-42b5-9160-efef3ec3447e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:07.418142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:07.419343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.419611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.419875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] PATH_INFO: `/v3/projects/d6c0a4e40e3e4e1f8d962e848e11d3c4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.421184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c0a4e40e3e4e1f8d962e848e11d3c4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:07.423424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:07.441835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:07.468847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:07.469765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.470228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.470551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] PATH_INFO: `/v3/projects/a6422b4f904b48d09b53fa960c54140a` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.472086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6422b4f904b48d09b53fa960c54140a)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:07.474040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:07.489992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c0a4e40e3e4e1f8d962e848e11d3c4', 'request_id': 'req-a3dc838f-3d5d-423e-8ab0-30992a055e32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.500053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c0a4e40e3e4e1f8d962e848e11d3c4', 'request_id': 'req-a3dc838f-3d5d-423e-8ab0-30992a055e32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.505858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c0a4e40e3e4e1f8d962e848e11d3c4', 'request_id': 'req-a3dc838f-3d5d-423e-8ab0-30992a055e32'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.520824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] The token cache is being invalidate because project d6c0a4e40e3e4e1f8d962e848e11d3c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:07.521169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.521816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.523492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1087/2192] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:07 2022] DELETE /identity/v3/projects/d6c0a4e40e3e4e1f8d962e848e11d3c4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:07.528821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:07.530173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.546467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6422b4f904b48d09b53fa960c54140a', 'request_id': 'req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.553112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6422b4f904b48d09b53fa960c54140a', 'request_id': 'req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.562332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6422b4f904b48d09b53fa960c54140a', 'request_id': 'req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.573848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] The token cache is being invalidate because project a6422b4f904b48d09b53fa960c54140a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:07.574145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.574840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bb5797f9-ffc7-46b3-9ef3-40c66b727a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.576643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1106/2193] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:07 2022] DELETE /identity/v3/projects/a6422b4f904b48d09b53fa960c54140a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:07.581603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d3ba-c5fb-4bec-9a57-4a9ca32873cb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.581897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d3ba-c5fb-4bec-9a57-4a9ca32873cb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.582220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d3ba-c5fb-4bec-9a57-4a9ca32873cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.598397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc838f-3d5d-423e-8ab0-30992a055e32 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:07.603391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8680d3ba-c5fb-4bec-9a57-4a9ca32873cb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:07.605833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8680d3ba-c5fb-4bec-9a57-4a9ca32873cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.642663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:07.662325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:07.663268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.663546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.663820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] PATH_INFO: `/v3/users/386f27696f3b4e2fb01c54228f273fd0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.665087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=386f27696f3b4e2fb01c54228f273fd0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:07.667190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:07.674595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1107/2194] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:07.682995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.683306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.683584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.706860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:07.708089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.717349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1108/2195] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:07.718681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:07.720593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '386f27696f3b4e2fb01c54228f273fd0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.721837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:07.732378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6010f-b069-418e-b6ea-066d85060fb2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:07.737674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '386f27696f3b4e2fb01c54228f273fd0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.738916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:07.739597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.739884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.740153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.740895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:07.741207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:07.743001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:07.747363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '386f27696f3b4e2fb01c54228f273fd0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.755526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '386f27696f3b4e2fb01c54228f273fd0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.756248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '386f27696f3b4e2fb01c54228f273fd0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:07.757042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1109/2196] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:07.761058 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:07.763293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.767961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1088/2197] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:07 2022] DELETE /identity/v3/users/386f27696f3b4e2fb01c54228f273fd0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:07.780837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2296-11d9-41ec-8215-b5b791620242 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:07.864712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:07.865403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:07.865858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:07.866044 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:07.869600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:07.869763 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:07.894029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:07.895622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:07.917186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b788-cd93-4341-a1d7-57c52bea2165 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:07.921614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1110/2198] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:07 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:18:07.926543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.017691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9a870-a521-461a-bc6c-67dc469f5c44 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.035681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.036453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.037057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.037394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.038156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.045249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.065560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.065878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.066299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.066617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] PATH_INFO: `/v3/users/56bc9a55f0f74abca0d8e8606907cffb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.069567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56bc9a55f0f74abca0d8e8606907cffb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.073905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.076587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1111/2199] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:07 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:18:08.080628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.096041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b8a0a-7dc5-4b6e-ada8-aca5369d18ec admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.124179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.125066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.125520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.125745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.126756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.128761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.150817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1112/2200] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:08.152344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.164654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0307ad-b52a-49d3-a1d6-2fbbce8a1fdc admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.167772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56bc9a55f0f74abca0d8e8606907cffb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.172071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.172775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.173062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.173332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.174077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.177318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.187662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56bc9a55f0f74abca0d8e8606907cffb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.192588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1113/2201] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:08.197755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.203508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56bc9a55f0f74abca0d8e8606907cffb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.214226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17667a5-3fcb-4d02-b236-38eb65f80c6b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.217628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56bc9a55f0f74abca0d8e8606907cffb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.218898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56bc9a55f0f74abca0d8e8606907cffb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.220487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210e3c9-6395-4a72-b289-cb9d18b26817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.221409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e3c9-6395-4a72-b289-cb9d18b26817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.221689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e3c9-6395-4a72-b289-cb9d18b26817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.221961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e3c9-6395-4a72-b289-cb9d18b26817 admin admin] PATH_INFO: `/v3/projects/3b0d9d5dc98e4aa2971a5f545019ed88/users/6e2bf1dff5fa42c2b4dc1491cef03d46/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.228557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1089/2202] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:07 2022] DELETE /identity/v3/users/56bc9a55f0f74abca0d8e8606907cffb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:08.230780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.232965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.235098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:08.241225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210e3c9-6395-4a72-b289-cb9d18b26817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0d9d5dc98e4aa2971a5f545019ed88, user_id=6e2bf1dff5fa42c2b4dc1491cef03d46, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.244376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210e3c9-6395-4a72-b289-cb9d18b26817 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.265042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1114/2203] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:08 2022] PUT /identity/v3/projects/3b0d9d5dc98e4aa2971a5f545019ed88/users/6e2bf1dff5fa42c2b4dc1491cef03d46/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:08.275622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc89c3-32b3-4382-807e-7b96331c9476 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.275911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc89c3-32b3-4382-807e-7b96331c9476 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.276187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc89c3-32b3-4382-807e-7b96331c9476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.295052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-68dc89c3-32b3-4382-807e-7b96331c9476 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:08.296308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-68dc89c3-32b3-4382-807e-7b96331c9476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.297117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:08.326126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670248d9-faf0-4a78-a201-34c3c6146d21 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.368273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1115/2204] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:08.370095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a6d1-111e-4f4e-9070-19d0e26c0fcd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.370623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a6d1-111e-4f4e-9070-19d0e26c0fcd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.371137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a6d1-111e-4f4e-9070-19d0e26c0fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.396623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.397879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.398418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.399504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] PATH_INFO: `/v3/projects/86bf32696b9942ca845d59170e24e09b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.400797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bf32696b9942ca845d59170e24e09b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.403497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.425121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1fd1a6d1-111e-4f4e-9070-19d0e26c0fcd None None] MFA Rules not processed for user `6e2bf1dff5fa42c2b4dc1491cef03d46`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:08.426682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1fd1a6d1-111e-4f4e-9070-19d0e26c0fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.462976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bf32696b9942ca845d59170e24e09b', 'request_id': 'req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.476352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bf32696b9942ca845d59170e24e09b', 'request_id': 'req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.485439 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bf32696b9942ca845d59170e24e09b', 'request_id': 'req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.501940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] The token cache is being invalidate because project 86bf32696b9942ca845d59170e24e09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:08.502257 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.502956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:08.504629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1090/2205] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:08 2022] DELETE /identity/v3/projects/86bf32696b9942ca845d59170e24e09b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:08.505602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.507149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.519678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1116/2206] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:08.521396 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.521690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.521969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.542307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:08.543579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.572929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be407fdc-2bf5-4408-b9ca-b948c6dc73f5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.577003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.577673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.577981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.578225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.580108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.608804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1117/2207] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:08.610141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.611528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.625335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800dc31-9e3e-492d-86b7-4129fa3844fd None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.626592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:08.630178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aff27-597a-47e7-8c51-0f4ed2e12d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.631539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aff27-597a-47e7-8c51-0f4ed2e12d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.631539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aff27-597a-47e7-8c51-0f4ed2e12d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.631774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aff27-597a-47e7-8c51-0f4ed2e12d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.632321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3aff27-597a-47e7-8c51-0f4ed2e12d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.634503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3aff27-597a-47e7-8c51-0f4ed2e12d36 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.656948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:08.667076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1118/2208] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:08.669032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.670520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.672482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4122a-76cb-437e-816c-f7927801499e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.673211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4122a-76cb-437e-816c-f7927801499e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.673916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4122a-76cb-437e-816c-f7927801499e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.682434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1091/2209] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:08.688880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.704511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f12e85-d086-4a8e-9c7e-378f346c6e25 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.707526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-80a4122a-76cb-437e-816c-f7927801499e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:08.709231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-80a4122a-76cb-437e-816c-f7927801499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:08.804277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.805345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.805626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.805902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.806929 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.809345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.817782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1119/2210] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:08.822983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4122a-76cb-437e-816c-f7927801499e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.835091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4122a-76cb-437e-816c-f7927801499e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.836895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1092/2211] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:08.842372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.843916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.844661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.844999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.845280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.846078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:08.846440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.848571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.853075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc093b7-34dc-43bf-bebc-a36556318a8b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.858334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.859070 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.859357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.859631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.860381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.862016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.863977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1120/2212] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:08.869104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.880973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1093/2213] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:08.881904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626295b5-5014-418f-a371-cce8b0fd92d4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.882021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.890588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.891355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.891642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.892207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.893584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.895137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032b4a1-9ef7-4e0a-afea-31dfa67cb6a2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:08.895914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.901572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:08.902842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:08.902973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:08.903247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] PATH_INFO: `/v3/projects/9b8fad6dcc6540a0b01d3198fc9fc3ef/users/a926611b23614ae3830a8406aa9591f5/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:08.923414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8fad6dcc6540a0b01d3198fc9fc3ef, user_id=a926611b23614ae3830a8406aa9591f5, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:08.927212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:08.937837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1121/2214] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:08.942332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:08.952243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1094/2215] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:08 2022] PUT /identity/v3/projects/9b8fad6dcc6540a0b01d3198fc9fc3ef/users/a926611b23614ae3830a8406aa9591f5/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:08.955980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.040499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47052-167d-4f99-9106-6e3f547f2198 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.045447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b984db-c187-455e-b9e4-07f3c2ec385f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.048650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.049361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.049648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.049948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.050517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.050689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.051244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.052143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.052423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.052621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.053158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.054937 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.070490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1095/2216] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:09.074345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.091189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1122/2217] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:08 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:18:09.099267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.106648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaf803-08ce-46e7-85ec-a9161fc50db6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.112866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.113771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.114049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.114316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] PATH_INFO: `/v3/projects/9b8fad6dcc6540a0b01d3198fc9fc3ef/users/a926611b23614ae3830a8406aa9591f5/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.116526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8fad6dcc6540a0b01d3198fc9fc3ef, user_id=a926611b23614ae3830a8406aa9591f5, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.119251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.120809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c9cf8-956d-4434-bc43-37269021172c admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.132129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.132822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.133104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.133394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.134122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.136801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.147623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1096/2218] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:09 2022] PUT /identity/v3/projects/9b8fad6dcc6540a0b01d3198fc9fc3ef/users/a926611b23614ae3830a8406aa9591f5/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:09.148056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.150054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1123/2219] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:09.151473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:09.154225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.165477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f761702-8a61-4cd4-b702-a60411c92dc6 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.166729 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:09.219033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:09.239815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a09829-e600-4367-a2f3-ef782b6231a7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.242219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25885a-c4be-4e61-824d-3d54610a3491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.242948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25885a-c4be-4e61-824d-3d54610a3491 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.243231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25885a-c4be-4e61-824d-3d54610a3491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.243506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25885a-c4be-4e61-824d-3d54610a3491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.244250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25885a-c4be-4e61-824d-3d54610a3491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.245516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.245912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25885a-c4be-4e61-824d-3d54610a3491 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.246574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.246964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.247520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] PATH_INFO: `/v3/projects/66931d9e9d4849ab9c78bb11624e3177` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.249174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66931d9e9d4849ab9c78bb11624e3177)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.251295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.259701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1124/2220] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:09.259701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.259701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.259701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.288490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] MFA Rules not processed for user `a926611b23614ae3830a8406aa9591f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:09.289848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:09.325361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66931d9e9d4849ab9c78bb11624e3177', 'request_id': 'req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:09.333861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66931d9e9d4849ab9c78bb11624e3177', 'request_id': 'req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:09.342100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66931d9e9d4849ab9c78bb11624e3177', 'request_id': 'req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:09.358594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] The token cache is being invalidate because project 66931d9e9d4849ab9c78bb11624e3177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:09.358948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:09.359554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:09.364584 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1097/2221] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:09 2022] DELETE /identity/v3/projects/66931d9e9d4849ab9c78bb11624e3177 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:09.365517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.367758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:09.418059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1125/2222] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:09.431911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.435902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:09.518050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ba7a3-3c46-4a9f-9b5a-0eb87d38d76d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.522885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea807bb2-ea21-47ad-98bd-e6d0aa85700d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.524247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991abf1-bb90-4d88-90e2-349fa0c7fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.525345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991abf1-bb90-4d88-90e2-349fa0c7fac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.525857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991abf1-bb90-4d88-90e2-349fa0c7fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.526202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991abf1-bb90-4d88-90e2-349fa0c7fac9 admin admin] PATH_INFO: `/v3/projects/c8482c30ccf84cdcbc9cf228dd2aebeb/users/ac020b78512843118992ead6272c7689/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.544858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991abf1-bb90-4d88-90e2-349fa0c7fac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8482c30ccf84cdcbc9cf228dd2aebeb, user_id=ac020b78512843118992ead6272c7689, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.548256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991abf1-bb90-4d88-90e2-349fa0c7fac9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.571019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1098/2223] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:09 2022] PUT /identity/v3/projects/c8482c30ccf84cdcbc9cf228dd2aebeb/users/ac020b78512843118992ead6272c7689/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:09.578344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.578948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.578948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.606079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.606779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.607107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.607393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.609311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:09.612716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] MFA Rules not processed for user `ac020b78512843118992ead6272c7689`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:09.614016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:09.623201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:09.652878 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:09.717003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.718832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.731891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1126/2224] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:18:09.741728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1099/2225] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:09.747520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.758653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d943e-fdf2-4d70-8e1d-c7e6547f256a service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.819352 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.820043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.820326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.820628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.821321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.823419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.852520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1127/2226] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:09 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:18:09.857682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.869250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c1b49-ed09-49b4-84b7-478f338d7a80 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.935568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.936244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:09.936531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:09.936812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:09.937526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:09.939583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:09.961513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1100/2227] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:09 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:09.966768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:09.978346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3938c6-b95d-4ef3-ab6b-21cda202fbd3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:09.993416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:09.999773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.000098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.000469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.001270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:10.003016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:10.019045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1128/2228] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:10.022214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:10.034139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd264ee-db4b-4e12-8e35-8bce4b5d7ed7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:10.046095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:10.047435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.047724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.048049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] PATH_INFO: `/v3/projects/e38192323b9f4dcbb943f4cd99913273/users/134e64f25d764239b1cbecde90fbeb30/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.065205 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38192323b9f4dcbb943f4cd99913273, user_id=134e64f25d764239b1cbecde90fbeb30, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:10.068161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:10.094646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1101/2229] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:10 2022] PUT /identity/v3/projects/e38192323b9f4dcbb943f4cd99913273/users/134e64f25d764239b1cbecde90fbeb30/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:10.098895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:10.110264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e857c-e1a6-42e5-88b2-37bd02348f92 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:10.172493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07a262-28b7-47a3-95ab-9ac0acd43b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:10.173186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a262-28b7-47a3-95ab-9ac0acd43b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.173465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a262-28b7-47a3-95ab-9ac0acd43b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.173739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a262-28b7-47a3-95ab-9ac0acd43b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.174470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07a262-28b7-47a3-95ab-9ac0acd43b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:10.176243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07a262-28b7-47a3-95ab-9ac0acd43b5e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:10.186983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1129/2230] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:10.191131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:10.202289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efcb79-13ec-4c68-88b1-af96f42a891e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:10.207074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:10.208094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.208445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.208723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] PATH_INFO: `/v3/projects/e38192323b9f4dcbb943f4cd99913273/users/134e64f25d764239b1cbecde90fbeb30/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.210773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38192323b9f4dcbb943f4cd99913273, user_id=134e64f25d764239b1cbecde90fbeb30, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:10.213536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:10.225263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1102/2231] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:10 2022] PUT /identity/v3/projects/e38192323b9f4dcbb943f4cd99913273/users/134e64f25d764239b1cbecde90fbeb30/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:10.237439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.237722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.238001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.264789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] MFA Rules not processed for user `134e64f25d764239b1cbecde90fbeb30`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:10.266005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:10.299736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:10.340569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1130/2232] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:10.352911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:10.355237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:10.428424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5947c4-f3cb-4e93-8788-bf8c50e7851e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:10.432294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71c44f-6d18-4158-be41-eb5bbb596b64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:10.433015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71c44f-6d18-4158-be41-eb5bbb596b64 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.433324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71c44f-6d18-4158-be41-eb5bbb596b64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.433582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71c44f-6d18-4158-be41-eb5bbb596b64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.443613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71c44f-6d18-4158-be41-eb5bbb596b64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:10.445104 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71c44f-6d18-4158-be41-eb5bbb596b64 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:10.456550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1103/2233] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:10.542620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:10.552708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6c58f-cc5d-4884-84fb-f5f6f4f7159e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:10.556861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:10.557529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:10.557790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:10.558064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:10.626075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:10.627576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:10.637419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1131/2234] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:12.411718 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:12.412282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:12.412432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:12.419504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:12.440740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] MFA Rules not processed for user `acbafaab99f848828c304ca5a45a55dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:12.442095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:12.504393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1104/2235] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:14.424841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:14.434818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1af1c-89d9-4b7d-864b-6fd6b938abcd service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:14.438128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8592de-3b5d-49a2-b8f3-c2bdf1451e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:14.438802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8592de-3b5d-49a2-b8f3-c2bdf1451e90 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:14.439086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8592de-3b5d-49a2-b8f3-c2bdf1451e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:14.439360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8592de-3b5d-49a2-b8f3-c2bdf1451e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:14.515376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8592de-3b5d-49a2-b8f3-c2bdf1451e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:14.516831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8592de-3b5d-49a2-b8f3-c2bdf1451e90 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:14.526888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1132/2236] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:15.489139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:15.557785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a240-ac98-4d78-b052-680e51569e0f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:15.563962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:15.565075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:15.565365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:15.565664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] PATH_INFO: `/v3/users/ac020b78512843118992ead6272c7689` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:15.566951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac020b78512843118992ead6272c7689)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:15.568920 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:15.621494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac020b78512843118992ead6272c7689'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:15.637459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac020b78512843118992ead6272c7689'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:15.647720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac020b78512843118992ead6272c7689'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:15.663283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac020b78512843118992ead6272c7689'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:15.664120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac020b78512843118992ead6272c7689'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:15.681260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1105/2237] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:15 2022] DELETE /identity/v3/users/ac020b78512843118992ead6272c7689 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:15.683103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:15.684341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:15.684341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:15.714524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:15.717558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:15.804696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1133/2238] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:15.815874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:15.817320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:15.857887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:15.899981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089cba9-6098-4b23-a4ec-dbac51b41b37 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:15.905136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:15.905932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:15.906291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:15.906642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:15.917456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:15.919376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:15.933468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1106/2239] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:16.247775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:16.249402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:16.257659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:16.265742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c712b9a-788c-402a-b8e5-3ea6483b0476 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:16.272410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:16.273808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:16.274090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:16.274558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] PATH_INFO: `/v3/projects/c8482c30ccf84cdcbc9cf228dd2aebeb` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:16.276251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8482c30ccf84cdcbc9cf228dd2aebeb)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:16.279514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:16.330893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8482c30ccf84cdcbc9cf228dd2aebeb', 'request_id': 'req-97339738-db7f-4fa1-9df3-276513c569a6'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.337963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8482c30ccf84cdcbc9cf228dd2aebeb', 'request_id': 'req-97339738-db7f-4fa1-9df3-276513c569a6'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.347389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8482c30ccf84cdcbc9cf228dd2aebeb', 'request_id': 'req-97339738-db7f-4fa1-9df3-276513c569a6'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.358696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] The token cache is being invalidate because project c8482c30ccf84cdcbc9cf228dd2aebeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:16.359011 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.359649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.361208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1134/2240] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:16 2022] DELETE /identity/v3/projects/c8482c30ccf84cdcbc9cf228dd2aebeb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:16.365329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:16.375896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8214b1-2e7d-491a-8ee2-12127cee79e9 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:16.436605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:16.437500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:16.437787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:16.438250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] PATH_INFO: `/v3/users/134e64f25d764239b1cbecde90fbeb30` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:16.439277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=134e64f25d764239b1cbecde90fbeb30)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:16.441440 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:16.493948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134e64f25d764239b1cbecde90fbeb30'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.508879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134e64f25d764239b1cbecde90fbeb30'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.518453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134e64f25d764239b1cbecde90fbeb30'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.530938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134e64f25d764239b1cbecde90fbeb30'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.531685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f12fec4c-38c1-47bb-8f6e-7f4414152389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134e64f25d764239b1cbecde90fbeb30'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.547664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1107/2241] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:16 2022] DELETE /identity/v3/users/134e64f25d764239b1cbecde90fbeb30 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:16.834789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:16.836317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:16.852862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339738-db7f-4fa1-9df3-276513c569a6 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:16.923483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:16.924684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:16.924958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:16.925229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] PATH_INFO: `/v3/projects/e38192323b9f4dcbb943f4cd99913273` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:16.926532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38192323b9f4dcbb943f4cd99913273)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:16.928837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:16.978362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38192323b9f4dcbb943f4cd99913273', 'request_id': 'req-8465986e-beef-462b-94de-5ad1ada237ba'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:16.985394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38192323b9f4dcbb943f4cd99913273', 'request_id': 'req-8465986e-beef-462b-94de-5ad1ada237ba'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:17.000610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38192323b9f4dcbb943f4cd99913273', 'request_id': 'req-8465986e-beef-462b-94de-5ad1ada237ba'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:17.013632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] The token cache is being invalidate because project e38192323b9f4dcbb943f4cd99913273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:17.014082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:17.014785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8465986e-beef-462b-94de-5ad1ada237ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:17.017049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1135/2242] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:16 2022] DELETE /identity/v3/projects/e38192323b9f4dcbb943f4cd99913273 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:17.022918 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.023437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.028849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.054499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:17.055996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:17.112484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:17.132219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1108/2243] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:17.140299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.140621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.140903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.160399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:17.161610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:17.171161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1136/2244] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:17.175738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.186753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7054f-57e1-4522-97eb-964abe4db198 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.191573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.192234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.192520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.192794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.193569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:17.193872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.195586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.208109 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1109/2245] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:17.212736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.222737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bfd75-af3e-466f-b601-731ca86c73f5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.227447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.228108 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.228458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.228671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.229416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.231464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.235730 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:17.270235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1137/2246] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:17.272631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.283416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29afc-036e-44f3-9022-99929c166193 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.361387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.362641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.363219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.363724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.364984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.368726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.393578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1110/2247] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:17.398363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.410624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd63b-b7d8-456e-a81a-0dbf99317596 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.416855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.417067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.417778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.417778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.418778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.420491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.431891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1138/2248] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:17.436858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.451140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb253-b3d2-4748-9949-bdf865e13c72 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.456340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e76f7-5b38-416c-a65f-bc8afed14bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.457026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e76f7-5b38-416c-a65f-bc8afed14bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.457311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e76f7-5b38-416c-a65f-bc8afed14bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.457646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e76f7-5b38-416c-a65f-bc8afed14bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.458451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e76f7-5b38-416c-a65f-bc8afed14bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.460226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e76f7-5b38-416c-a65f-bc8afed14bfb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.475683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1111/2249] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:17.477977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.491669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845e271-543c-44fe-afa5-950257c6b5db admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.497672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.498543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.499160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.500055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] PATH_INFO: `/v3/projects/fb4223fcacdc417badacd490658538b3/users/cc4b08b985284bd888f82219b11b6800/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.522684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4223fcacdc417badacd490658538b3, user_id=cc4b08b985284bd888f82219b11b6800, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.524915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.562157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1139/2250] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:17 2022] PUT /identity/v3/projects/fb4223fcacdc417badacd490658538b3/users/cc4b08b985284bd888f82219b11b6800/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:17.574852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.575395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.575488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.612824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] MFA Rules not processed for user `cc4b08b985284bd888f82219b11b6800`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:17.614291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:17.719595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1112/2251] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:17.724141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.834770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c52ee4-d81c-4608-a8cf-8481451b5f39 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.842042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.843829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.843829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.844438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.845212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.847240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:17.877773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1140/2252] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:17.881854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:17.896121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca990cc-0a96-4db0-a589-1c7ef3fb905a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:17.964580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:17.981332 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:17.982026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:17.982312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:17.982595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:17.983358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:17.985329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:18.006620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1113/2253] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:17 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:18.011304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:18.022358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbc7-5d59-4387-8514-a87ad6338eae admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:18.031172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:18.033732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:18.034028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:18.034419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:18.035510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:18.036890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:18.050816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1141/2254] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:18.053717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:18.064742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eecf76-37c8-4303-9994-42f9298cfbda admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:18.069843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:18.070811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:18.071097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:18.071372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] PATH_INFO: `/v3/projects/2fd90283b47243c2a563ffe86011bdba/users/dc00c8344eda48a2bd138f563d687c76/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:18.087983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd90283b47243c2a563ffe86011bdba, user_id=dc00c8344eda48a2bd138f563d687c76, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:18.091242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:18.105806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1114/2255] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:18 2022] PUT /identity/v3/projects/2fd90283b47243c2a563ffe86011bdba/users/dc00c8344eda48a2bd138f563d687c76/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:18.110167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:18.122062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f58c2f-23e8-4c99-945e-f31900bcf938 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:18.186082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780b65c-de95-4c6a-bf5e-3f20a78b66a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:18.186794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b65c-de95-4c6a-bf5e-3f20a78b66a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:18.187081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b65c-de95-4c6a-bf5e-3f20a78b66a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:18.187358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b65c-de95-4c6a-bf5e-3f20a78b66a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:18.188097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780b65c-de95-4c6a-bf5e-3f20a78b66a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:18.189727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780b65c-de95-4c6a-bf5e-3f20a78b66a9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:18.201209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1142/2256] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:18.205851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:18.217785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce57cc-d6e1-4adc-be5e-01a03945a361 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:18.222855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:18.223785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:18.224066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:18.224345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] PATH_INFO: `/v3/projects/2fd90283b47243c2a563ffe86011bdba/users/dc00c8344eda48a2bd138f563d687c76/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:18.226493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd90283b47243c2a563ffe86011bdba, user_id=dc00c8344eda48a2bd138f563d687c76, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:18.229236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:18.244832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1115/2257] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:18 2022] PUT /identity/v3/projects/2fd90283b47243c2a563ffe86011bdba/users/dc00c8344eda48a2bd138f563d687c76/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:18.256216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:18.256781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:18.257164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:18.305437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] MFA Rules not processed for user `dc00c8344eda48a2bd138f563d687c76`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:18.306676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:18.324576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:18.390684 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1143/2258] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:18.401625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:18.404113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:18.478026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255f0bc-7604-4fcf-8479-865940e55eeb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:18.482756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27d234-7e25-42d4-a567-f3225e1d3baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:18.483867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d234-7e25-42d4-a567-f3225e1d3baf service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:18.484172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d234-7e25-42d4-a567-f3225e1d3baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:18.484491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d234-7e25-42d4-a567-f3225e1d3baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:18.554769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27d234-7e25-42d4-a567-f3225e1d3baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:18.556228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27d234-7e25-42d4-a567-f3225e1d3baf service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:18.566338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1116/2259] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:21.228790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:21.294378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf97d09-173d-4803-9d4b-0f2a9bfc5071 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:21.299553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:21.300465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:21.300743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:21.301019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] PATH_INFO: `/v3/users/cc4b08b985284bd888f82219b11b6800` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:21.302281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4b08b985284bd888f82219b11b6800)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:21.304288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:21.356311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4b08b985284bd888f82219b11b6800'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:21.368696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4b08b985284bd888f82219b11b6800'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:21.378029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4b08b985284bd888f82219b11b6800'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:21.386514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4b08b985284bd888f82219b11b6800'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:21.387404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4b08b985284bd888f82219b11b6800'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:21.394527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1144/2260] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:21 2022] DELETE /identity/v3/users/cc4b08b985284bd888f82219b11b6800 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:21.399126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:21.399567 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:21.399738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:21.419142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:21.420391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:21.471345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:21.485714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1117/2261] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:21.496737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:21.498390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:21.531831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:21.565714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61952c07-80e0-4dd5-bc37-0efd8808a041 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:21.571775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:21.572524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:21.572876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:21.573258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:21.585711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:21.587496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:21.599573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1145/2262] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:21.916857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:21.918499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:21.932106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c7251-8f68-4edd-97c1-8cfac3d0060f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:21.938103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:21.939081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:21.939366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:21.940428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] PATH_INFO: `/v3/projects/fb4223fcacdc417badacd490658538b3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:21.942242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4223fcacdc417badacd490658538b3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:21.944469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:21.995823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4223fcacdc417badacd490658538b3', 'request_id': 'req-ad06c900-1656-45f3-b0dc-6b456d0ea197'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.001766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4223fcacdc417badacd490658538b3', 'request_id': 'req-ad06c900-1656-45f3-b0dc-6b456d0ea197'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.008410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4223fcacdc417badacd490658538b3', 'request_id': 'req-ad06c900-1656-45f3-b0dc-6b456d0ea197'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.022502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] The token cache is being invalidate because project fb4223fcacdc417badacd490658538b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:22.022896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.023524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.025603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1118/2263] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:21 2022] DELETE /identity/v3/projects/fb4223fcacdc417badacd490658538b3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:22.029774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:22.040495 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fed338-67e9-44aa-9869-f3c939d294a6 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:22.104579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:22.105800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.106096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.106374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] PATH_INFO: `/v3/users/dc00c8344eda48a2bd138f563d687c76` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.107711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc00c8344eda48a2bd138f563d687c76)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:22.109675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:22.157031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc00c8344eda48a2bd138f563d687c76'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.169232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc00c8344eda48a2bd138f563d687c76'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.177356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc00c8344eda48a2bd138f563d687c76'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.185148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc00c8344eda48a2bd138f563d687c76'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.185820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d80a45c0-8293-49a3-abec-373452164ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc00c8344eda48a2bd138f563d687c76'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.191854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1146/2264] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:22 2022] DELETE /identity/v3/users/dc00c8344eda48a2bd138f563d687c76 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:22.473781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:22.475359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:22.500811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c900-1656-45f3-b0dc-6b456d0ea197 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:22.567185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:22.567970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.568272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.568541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] PATH_INFO: `/v3/projects/2fd90283b47243c2a563ffe86011bdba` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.569734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd90283b47243c2a563ffe86011bdba)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:22.571691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:22.623984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd90283b47243c2a563ffe86011bdba', 'request_id': 'req-436b4684-34b8-4139-b297-3a4f99cc0082'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.629284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd90283b47243c2a563ffe86011bdba', 'request_id': 'req-436b4684-34b8-4139-b297-3a4f99cc0082'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.635494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd90283b47243c2a563ffe86011bdba', 'request_id': 'req-436b4684-34b8-4139-b297-3a4f99cc0082'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.649936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] The token cache is being invalidate because project 2fd90283b47243c2a563ffe86011bdba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:22.650373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.651145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-436b4684-34b8-4139-b297-3a4f99cc0082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:22.653909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1119/2265] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:22 2022] DELETE /identity/v3/projects/2fd90283b47243c2a563ffe86011bdba => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:22.658440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.658979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.659088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.678546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:22.679954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:22.741810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:22.760212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1147/2266] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:22.768080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.768370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.768684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.782613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:22.797365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:22.798628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:22.807226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1120/2267] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:22.811921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:22.822226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b8283-00bb-4d43-85d5-1200440cf076 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:22.827802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:22.828477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.828765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.829066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.829779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:22.830084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:22.831772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:22.850767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1148/2268] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:18:22.854843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:22.864965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2859dcc-29fb-4804-b4a5-1fd0c8d51458 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:22.870131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:22.871003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.871341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.871677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.872631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:22.874483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:22.908958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1121/2269] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:22 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:22.913166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:22.924797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fba14-bfa8-4a12-bf4f-b609b305a01f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:22.983201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:22.983891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:22.984174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:22.984449 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:22.985163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:22.987145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.010629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1149/2270] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:22 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:23.012004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.023089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3303454-5d00-469c-b04e-c603c04a3c15 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.028245 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.028915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.029194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.030829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.030829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.031836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.043458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1122/2271] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:23.047679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.058685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220a53-fa33-4995-8714-e620ba02f9fc admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.063999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b66de-f9a4-412f-8b45-99ef565a5a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.064680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b66de-f9a4-412f-8b45-99ef565a5a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.064964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b66de-f9a4-412f-8b45-99ef565a5a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.065237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b66de-f9a4-412f-8b45-99ef565a5a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.066007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b66de-f9a4-412f-8b45-99ef565a5a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.067649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b66de-f9a4-412f-8b45-99ef565a5a49 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.079309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1150/2272] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:23.083526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.094983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd50064-7ed6-4c8c-a25f-d57e50902aa0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.100204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.101097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.101365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.101639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] PATH_INFO: `/v3/projects/4af4de1c6cf449299860e01ac814a1ca/users/216e92c8e045471596065f3c08eff4f4/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.130075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af4de1c6cf449299860e01ac814a1ca, user_id=216e92c8e045471596065f3c08eff4f4, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.134403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.149163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1123/2273] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:23 2022] PUT /identity/v3/projects/4af4de1c6cf449299860e01ac814a1ca/users/216e92c8e045471596065f3c08eff4f4/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:23.160795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.161060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.161370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.192419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] MFA Rules not processed for user `216e92c8e045471596065f3c08eff4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:23.193650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:23.261603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1151/2274] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:23.266671 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.332443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bfbde-dfc4-4685-9d01-16b3f2812590 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.337141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.337808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.338093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.338342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.339273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.341878 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.372874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1124/2275] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:23 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:23.378783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.392306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7701ec-e65e-4907-b97b-7139cc93dd9c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.442513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:23.461465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.462147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.462428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.462720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.463426 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.466347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.487711 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1152/2276] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:23 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:23.492442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.506790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3dee-05a8-4821-a52a-0c7c0947270e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.512359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.513035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.513312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.513582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.514320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.515967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.527862 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1125/2277] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:23.532203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.543418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e103e48-bc46-45e2-82ff-f3308eeb6b60 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.548507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b942e5-1449-4131-ae25-935974c6dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.549251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b942e5-1449-4131-ae25-935974c6dad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.549521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b942e5-1449-4131-ae25-935974c6dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.549836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b942e5-1449-4131-ae25-935974c6dad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.550572 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b942e5-1449-4131-ae25-935974c6dad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.552217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b942e5-1449-4131-ae25-935974c6dad9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.575161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1153/2278] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:23.579716 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.590667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099485ae-d9b3-4c31-a423-28278cba5ceb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.595303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.596192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.596476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.596747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] PATH_INFO: `/v3/projects/7e532b0558f9430192447b16da1ae449/users/8a0522dcf7f8477584525783b0e6132d/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.614399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e532b0558f9430192447b16da1ae449, user_id=8a0522dcf7f8477584525783b0e6132d, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.617172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.636186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1126/2279] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:23 2022] PUT /identity/v3/projects/7e532b0558f9430192447b16da1ae449/users/8a0522dcf7f8477584525783b0e6132d/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:23.647402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.647692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.647972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.678114 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] MFA Rules not processed for user `8a0522dcf7f8477584525783b0e6132d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:23.679381 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:23.752416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1154/2280] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:23.757259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.816954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:23.830730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225141-3919-43eb-9c86-1bc677d05f99 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.836509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.837175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.837461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.837737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.838477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.841161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.874463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1127/2281] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:23 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:23.875698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:23.897996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800965bc-5ed7-4b5f-a740-640ebc1fadcd None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:23.966061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:23.966817 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:23.967100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:23.967376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:23.968089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:23.970045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:23.993785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1155/2282] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:23 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:23.997953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:24.008521 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399a76-c6f7-4868-ab6a-012698503e58 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:24.014389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:24.015081 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:24.015362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:24.015634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:24.016391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:24.017996 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:24.030564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1128/2283] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:24.034974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:24.046893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a297-56ae-4f30-90d0-014bff7b8783 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:24.052631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:24.053550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:24.053825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:24.054098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] PATH_INFO: `/v3/projects/69c2c98f3a15468580f7ca478855c980/users/41089cfba0d540baae550f5d7d6cfc26/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:24.071029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c2c98f3a15468580f7ca478855c980, user_id=41089cfba0d540baae550f5d7d6cfc26, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:24.073785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:24.100312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1156/2284] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:24 2022] PUT /identity/v3/projects/69c2c98f3a15468580f7ca478855c980/users/41089cfba0d540baae550f5d7d6cfc26/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:24.104288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:24.114868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54dc02-e99b-4c41-bfcd-fa2bf4663fc3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:24.176484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156cd7d-e162-4aa6-b1a8-02fb779c0d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:24.176905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156cd7d-e162-4aa6-b1a8-02fb779c0d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:24.177201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156cd7d-e162-4aa6-b1a8-02fb779c0d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:24.177587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156cd7d-e162-4aa6-b1a8-02fb779c0d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:24.178470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156cd7d-e162-4aa6-b1a8-02fb779c0d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:24.180178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156cd7d-e162-4aa6-b1a8-02fb779c0d7d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:24.192708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1129/2285] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:24.199605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:24.210500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6269d-feda-4e45-9a5a-7a77277a67fe admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:24.216636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:24.217517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:24.217800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:24.218109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] PATH_INFO: `/v3/projects/69c2c98f3a15468580f7ca478855c980/users/41089cfba0d540baae550f5d7d6cfc26/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:24.220289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c2c98f3a15468580f7ca478855c980, user_id=41089cfba0d540baae550f5d7d6cfc26, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:24.223091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:24.235021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1157/2286] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:24 2022] PUT /identity/v3/projects/69c2c98f3a15468580f7ca478855c980/users/41089cfba0d540baae550f5d7d6cfc26/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:24.245458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:24.245741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:24.246075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:24.280137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] MFA Rules not processed for user `41089cfba0d540baae550f5d7d6cfc26`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:24.281367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:24.382213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1130/2287] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:24.391443 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:24.393198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:24.407714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:24.476592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7dc8-2983-4bed-8987-c48a779c1418 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:24.482173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:24.483064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:24.483361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:24.483603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:24.595694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:24.597260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:24.608369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1158/2288] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:26.190773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:26.201774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e063bf-b34e-48f9-90b4-b7cb6b27a03f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:26.205870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:26.206710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:26.207009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:26.207283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:26.262479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:26.277456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:26.279164 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:26.290275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1131/2289] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:26.971744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:26.981019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83c38-5d22-4564-b434-14b825ccfc00 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:26.985111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21044c-efd3-4423-96bd-5d9f73b58528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:26.985759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21044c-efd3-4423-96bd-5d9f73b58528 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:26.986042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21044c-efd3-4423-96bd-5d9f73b58528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:26.986317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21044c-efd3-4423-96bd-5d9f73b58528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:26.996634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21044c-efd3-4423-96bd-5d9f73b58528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:26.998082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21044c-efd3-4423-96bd-5d9f73b58528 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:27.008078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1159/2290] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:36.285585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:37.003751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=52 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:43.565098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:43.690795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066dc7e-6fce-4454-882f-147f0950be19 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:43.695925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:43.696826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:43.697116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:43.697413 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] PATH_INFO: `/v3/users/216e92c8e045471596065f3c08eff4f4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:43.698655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] RBAC: Authorizing `identity:delete_user(user_id=216e92c8e045471596065f3c08eff4f4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:43.700819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:43.768657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216e92c8e045471596065f3c08eff4f4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:43.784579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216e92c8e045471596065f3c08eff4f4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:43.794461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216e92c8e045471596065f3c08eff4f4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:43.804326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216e92c8e045471596065f3c08eff4f4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:43.805327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216e92c8e045471596065f3c08eff4f4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:43.815350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1132/2291] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:43 2022] DELETE /identity/v3/users/216e92c8e045471596065f3c08eff4f4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:43.819970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:43.820492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:43.820580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:43.881729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:43.883261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:43.988706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1160/2292] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:43.999123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:44.001874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:44.090133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:44.172208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619438c2-343c-4a5f-95c9-61c50eda152e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:44.177030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:44.177848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:44.178130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:44.178453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:44.190989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:44.192489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:44.204293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1133/2293] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:44.564342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:44.565877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:44.579349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d341a-50a4-4d4e-aa2e-385ef445dac6 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:44.584257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:44.585201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:44.585523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:44.585898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] PATH_INFO: `/v3/projects/4af4de1c6cf449299860e01ac814a1ca` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:44.587197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af4de1c6cf449299860e01ac814a1ca)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:44.589224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:44.651227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af4de1c6cf449299860e01ac814a1ca', 'request_id': 'req-96b9be41-3274-4e8d-ab10-727444f3603e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.658485 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af4de1c6cf449299860e01ac814a1ca', 'request_id': 'req-96b9be41-3274-4e8d-ab10-727444f3603e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.667560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af4de1c6cf449299860e01ac814a1ca', 'request_id': 'req-96b9be41-3274-4e8d-ab10-727444f3603e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.679038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] The token cache is being invalidate because project 4af4de1c6cf449299860e01ac814a1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:44.679327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.679981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.681647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1161/2294] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:44 2022] DELETE /identity/v3/projects/4af4de1c6cf449299860e01ac814a1ca => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:44.685901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:44.697036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b75c-c9ab-4b10-97af-32312b247c82 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:44.756927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:44.757791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:44.758076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:44.758348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] PATH_INFO: `/v3/users/8a0522dcf7f8477584525783b0e6132d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:44.759766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0522dcf7f8477584525783b0e6132d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:44.761773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:44.820994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0522dcf7f8477584525783b0e6132d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.835003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0522dcf7f8477584525783b0e6132d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.849467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0522dcf7f8477584525783b0e6132d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.861258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0522dcf7f8477584525783b0e6132d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.862378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0522dcf7f8477584525783b0e6132d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:44.872156 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1134/2295] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:44 2022] DELETE /identity/v3/users/8a0522dcf7f8477584525783b0e6132d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:45.151917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:45.154017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:45.171552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9be41-3274-4e8d-ab10-727444f3603e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:45.240177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:45.247568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:45.248473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:45.248997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:45.249597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] PATH_INFO: `/v3/projects/7e532b0558f9430192447b16da1ae449` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:45.251293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e532b0558f9430192447b16da1ae449)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:45.253828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:45.312808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e532b0558f9430192447b16da1ae449', 'request_id': 'req-2b69937e-08d0-4105-a555-2df4cc7a8cb4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.322552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e532b0558f9430192447b16da1ae449', 'request_id': 'req-2b69937e-08d0-4105-a555-2df4cc7a8cb4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.335822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e532b0558f9430192447b16da1ae449', 'request_id': 'req-2b69937e-08d0-4105-a555-2df4cc7a8cb4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.350754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] The token cache is being invalidate because project 7e532b0558f9430192447b16da1ae449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:45.351127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.351869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.354836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1162/2296] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:45 2022] DELETE /identity/v3/projects/7e532b0558f9430192447b16da1ae449 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:45.358323 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:45.360131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:45.376843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ae070-afd0-4d8a-8dfc-91d5a8ddc51a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:45.398424 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:45.439658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:45.440564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:45.440841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:45.441127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] PATH_INFO: `/v3/users/41089cfba0d540baae550f5d7d6cfc26` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:45.442505 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41089cfba0d540baae550f5d7d6cfc26)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:45.444705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:45.493100 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41089cfba0d540baae550f5d7d6cfc26'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.505990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41089cfba0d540baae550f5d7d6cfc26'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.519891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41089cfba0d540baae550f5d7d6cfc26'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.528833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41089cfba0d540baae550f5d7d6cfc26'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.529568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a5b56ce1-9a53-4f54-a180-a33a768b5e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41089cfba0d540baae550f5d7d6cfc26'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.536682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1135/2297] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:45 2022] DELETE /identity/v3/users/41089cfba0d540baae550f5d7d6cfc26 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:45.819548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:45.821211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:45.835078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69937e-08d0-4105-a555-2df4cc7a8cb4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:45.907840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:45.909319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:45.909687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:45.910094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] PATH_INFO: `/v3/projects/69c2c98f3a15468580f7ca478855c980` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:45.912051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c2c98f3a15468580f7ca478855c980)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:45.914981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:45.967739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c2c98f3a15468580f7ca478855c980', 'request_id': 'req-bba07aac-9b77-4362-828c-5b73f2360c7b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.974979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c2c98f3a15468580f7ca478855c980', 'request_id': 'req-bba07aac-9b77-4362-828c-5b73f2360c7b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:45.984784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c2c98f3a15468580f7ca478855c980', 'request_id': 'req-bba07aac-9b77-4362-828c-5b73f2360c7b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.001641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] The token cache is being invalidate because project 69c2c98f3a15468580f7ca478855c980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:46.001934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.002645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-bba07aac-9b77-4362-828c-5b73f2360c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.004377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1163/2298] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:45 2022] DELETE /identity/v3/projects/69c2c98f3a15468580f7ca478855c980 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:46.010036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.010564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.010564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.060450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:46.061696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.124055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1136/2299] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:46.135645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.135935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.136209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.155708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:46.156957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.166238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1164/2300] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:46.170802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.181634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17923c84-1be0-4707-8a38-429a41fa2468 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.188105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.188776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.189060 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.189366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.190134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:46.190442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.192150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.205968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1137/2301] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:46.210925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.220974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba752-31fc-4e78-a93e-123c3f24757d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.225710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.226369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.226650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.228331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.228331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.229576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.238553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:46.252726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.254651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.259941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1165/2302] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:46.263990 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.267932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02548cd-d8e2-43f5-9165-c26d9f71da4c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.271818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:46.335224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf5ab4-9228-4dcd-9c5e-cf7f75d82177 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.341594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.342403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.342688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.342997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.343701 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.343809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.344704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.345112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.345417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] PATH_INFO: `/v3/users/eb2ccb70920e48e580d014b6eaad238a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.345644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.346656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2ccb70920e48e580d014b6eaad238a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.348816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.372506 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1166/2303] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:46.384411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.403104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdff6ae-593c-4033-9dd0-cebf18be4a61 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.407811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.408487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.408773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.409048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.409783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.411649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.412954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2ccb70920e48e580d014b6eaad238a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.426069 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1167/2304] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:46.429533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.433351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.435689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2ccb70920e48e580d014b6eaad238a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.445410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2ccb70920e48e580d014b6eaad238a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.452550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8912858-6209-4403-9fc5-a9f372b19f1f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.458453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.459408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.459745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.460031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] PATH_INFO: `/v3/users/6e2bf1dff5fa42c2b4dc1491cef03d46` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.460149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2ccb70920e48e580d014b6eaad238a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.460908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2ccb70920e48e580d014b6eaad238a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.461260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2bf1dff5fa42c2b4dc1491cef03d46)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.463255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.471975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1138/2305] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:46 2022] DELETE /identity/v3/users/eb2ccb70920e48e580d014b6eaad238a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:46.472907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.480232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.533394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2bf1dff5fa42c2b4dc1491cef03d46'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.550842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2bf1dff5fa42c2b4dc1491cef03d46'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.560415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2bf1dff5fa42c2b4dc1491cef03d46'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.569136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2bf1dff5fa42c2b4dc1491cef03d46'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.570271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7e8bbed-18f7-41be-a19e-223c46a6f75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2bf1dff5fa42c2b4dc1491cef03d46'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:46.579476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1168/2306] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:46 2022] DELETE /identity/v3/users/6e2bf1dff5fa42c2b4dc1491cef03d46 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:46.581576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d312-f977-46c2-8d61-568508e5b656 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.582120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d312-f977-46c2-8d61-568508e5b656 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.582639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d312-f977-46c2-8d61-568508e5b656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.585262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8ebd9-ca19-47a2-9603-783676e87360 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.605771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-af47d312-f977-46c2-8d61-568508e5b656 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:46.607612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-af47d312-f977-46c2-8d61-568508e5b656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.627560 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:46.650585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:46.667753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac80a55-ddde-4665-b3a6-95fbca38a6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.668738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80a55-ddde-4665-b3a6-95fbca38a6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.669272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80a55-ddde-4665-b3a6-95fbca38a6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.669640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80a55-ddde-4665-b3a6-95fbca38a6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.670633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac80a55-ddde-4665-b3a6-95fbca38a6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.673016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac80a55-ddde-4665-b3a6-95fbca38a6cc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.689940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1139/2307] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:46.689940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.689940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.689940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.698751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1169/2308] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:46.701089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.701438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.708137 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.728953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:46.730158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.733055 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:46.734254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.742007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1170/2309] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:46.742007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.742007 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.753886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baafd5c-92df-41e3-a53f-f9b85cf71eeb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.758734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.759636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.759915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.760190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] PATH_INFO: `/v3/projects/90f6878fdaaf43b58b92bea357331d2e/users/23c7ecdb284d4ad0bda072a87ad8984c/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.778490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f6878fdaaf43b58b92bea357331d2e, user_id=23c7ecdb284d4ad0bda072a87ad8984c, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.781062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.806482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1171/2310] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:46 2022] PUT /identity/v3/projects/90f6878fdaaf43b58b92bea357331d2e/users/23c7ecdb284d4ad0bda072a87ad8984c/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:46.806482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.808648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.817516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1140/2311] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:46.822598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:46.824406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:46.896745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488f30e-c967-4d11-bdf7-310a569fbf75 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.903218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6adb8-5e3d-49a2-8e24-72f0777dd267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.903968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6adb8-5e3d-49a2-8e24-72f0777dd267 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.904256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6adb8-5e3d-49a2-8e24-72f0777dd267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.904534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6adb8-5e3d-49a2-8e24-72f0777dd267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.905912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb4f9e-dc3d-4f73-9af7-6a2c7a565c98 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:46.910535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:46.911427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:46.911427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:46.912041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:46.987916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6adb8-5e3d-49a2-8e24-72f0777dd267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:46.989440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6adb8-5e3d-49a2-8e24-72f0777dd267 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:46.999593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:47.000470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1172/2312] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:47.001526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:47.001870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.002169 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.002513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.015854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1141/2313] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:47.016829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:47.023387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:47.023548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:47.024749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:47.037939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1173/2314] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:47.044241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:47.050609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:47.058577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1970c-3ff4-4d96-88f6-826c3abdcdce None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:47.064319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:47.065136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.065554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.065832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.082407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:47.084806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:47.105472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1174/2315] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:47.123541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333ef2-3e3d-46d8-b02b-bb909a21bb3d service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:47.131908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:47.132695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.133018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.133348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.211866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:47.214315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:47.231672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1142/2316] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:18:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:18:47.534854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:47.536316 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:47.556588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c0df0-1d34-44bf-85de-5f14f71feaf1 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:47.570828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:47.571876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.572334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.572465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] PATH_INFO: `/v3/projects/af2928ac8a8e4e3a8bc0b2fcc7c81265` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.578997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2928ac8a8e4e3a8bc0b2fcc7c81265)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:47.581100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:47.678791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:47.678791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:47.713766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:47.720295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9c2e7-47ea-4d01-9414-9e63ee07174d service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:47.737027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2928ac8a8e4e3a8bc0b2fcc7c81265', 'request_id': 'req-f605fc1a-4845-4f2b-bfcd-9b54431944c4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:47.753311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2928ac8a8e4e3a8bc0b2fcc7c81265', 'request_id': 'req-f605fc1a-4845-4f2b-bfcd-9b54431944c4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:47.775697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2928ac8a8e4e3a8bc0b2fcc7c81265', 'request_id': 'req-f605fc1a-4845-4f2b-bfcd-9b54431944c4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:47.801003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] The token cache is being invalidate because project af2928ac8a8e4e3a8bc0b2fcc7c81265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:47.801327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:47.802098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f605fc1a-4845-4f2b-bfcd-9b54431944c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:47.805768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1175/2317] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:47 2022] DELETE /identity/v3/projects/af2928ac8a8e4e3a8bc0b2fcc7c81265 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:47.811510 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3403e-9c8e-40fb-91e4-3d21037da72e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.811757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3403e-9c8e-40fb-91e4-3d21037da72e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.812031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3403e-9c8e-40fb-91e4-3d21037da72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.850597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-50e3403e-9c8e-40fb-91e4-3d21037da72e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:47.853228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-50e3403e-9c8e-40fb-91e4-3d21037da72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:47.870680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:47.873416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.875723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.876202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] PATH_INFO: `/v3/projects/3b0d9d5dc98e4aa2971a5f545019ed88` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.878566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1176/2318] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:47.883556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0d9d5dc98e4aa2971a5f545019ed88)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:47.885977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:47.886959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:47.887610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:47.888207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:47.921384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:47.923517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:47.946075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1177/2319] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:47.951913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:47.969194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42a059-a28d-4284-a0cb-6c18cc39d88a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.011357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0d9d5dc98e4aa2971a5f545019ed88', 'request_id': 'req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.022286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0d9d5dc98e4aa2971a5f545019ed88', 'request_id': 'req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.025146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:48.038252 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0d9d5dc98e4aa2971a5f545019ed88', 'request_id': 'req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.062090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] The token cache is being invalidate because project 3b0d9d5dc98e4aa2971a5f545019ed88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:48.062392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.064006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.073000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1143/2320] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:47 2022] DELETE /identity/v3/projects/3b0d9d5dc98e4aa2971a5f545019ed88 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:48.074785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.076761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:48.114932 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.116315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.117129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.117694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.118646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:48.119386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.122055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.149807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1178/2321] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:48.155828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.159267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:48.164663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc5914-508a-4ef5-91c8-e9d70dd4dd67 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.169991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.170927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.171409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.171623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] PATH_INFO: `/v3/users/a926611b23614ae3830a8406aa9591f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.173117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a926611b23614ae3830a8406aa9591f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.175114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.187984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dbad8-b28a-441a-8360-563e35a87fe4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.200199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.201083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.201356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.203105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.204088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.211372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.244861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a926611b23614ae3830a8406aa9591f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.260420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a926611b23614ae3830a8406aa9591f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.272075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a926611b23614ae3830a8406aa9591f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.275384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1179/2322] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:48 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:48.280647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.286397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a926611b23614ae3830a8406aa9591f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.287250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a926611b23614ae3830a8406aa9591f5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:48.296235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1144/2323] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:18:48 2022] DELETE /identity/v3/users/a926611b23614ae3830a8406aa9591f5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:48.427585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b400c5-ad7e-4d8a-bd66-c40731ddd6d2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.483307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:48.537623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.538647 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.539472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.539472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.540894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.543708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.576564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1180/2324] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:48 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:18:48.581119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.582574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:48.605039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec815-1196-407c-917f-e1c438db3a58 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.611782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.612559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.612913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.613260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.614142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.617342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.638193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.644110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1145/2325] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:48.660171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008738a4-bfb9-43c1-a3b8-e282d9be0bb7 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.669643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f37638-bbae-445e-abe9-ce28001aa95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.670492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f37638-bbae-445e-abe9-ce28001aa95f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.670960 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f37638-bbae-445e-abe9-ce28001aa95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.671331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f37638-bbae-445e-abe9-ce28001aa95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.672312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f37638-bbae-445e-abe9-ce28001aa95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.674152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f37638-bbae-445e-abe9-ce28001aa95f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.712349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.725468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aea610-9aac-459a-afda-cbcb51c59f40 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:48.726061 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1181/2326] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:48.735860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:48.739636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.739636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.739636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] PATH_INFO: `/v3/projects/f68ef0ce863f43cdab4afcba5329f925/users/d25025d4a28e451fb4e60b5a1e97ce4c/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.791834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68ef0ce863f43cdab4afcba5329f925, user_id=d25025d4a28e451fb4e60b5a1e97ce4c, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:48.794813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:48.830286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1146/2327] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:48 2022] PUT /identity/v3/projects/f68ef0ce863f43cdab4afcba5329f925/users/d25025d4a28e451fb4e60b5a1e97ce4c/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:48.842959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:48.843323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:48.843675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:48.930309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] MFA Rules not processed for user `d25025d4a28e451fb4e60b5a1e97ce4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:48.931527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:48.982793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:48.985183 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:49.067899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:49.100858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1182/2328] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:49.114685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:49.117006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:49.149484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cf56f-cf68-47f7-bff7-2afa93179d33 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:49.155223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:49.155889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:49.156442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:49.156442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] PATH_INFO: `/v3/projects/9b8fad6dcc6540a0b01d3198fc9fc3ef` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:49.157681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8fad6dcc6540a0b01d3198fc9fc3ef)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:49.159742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:49.219473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8a484-933d-47f3-a7e9-1d492eac19a9 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:49.240880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8fad6dcc6540a0b01d3198fc9fc3ef', 'request_id': 'req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:49.251198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8fad6dcc6540a0b01d3198fc9fc3ef', 'request_id': 'req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:49.261299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8fad6dcc6540a0b01d3198fc9fc3ef', 'request_id': 'req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:49.286029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] The token cache is being invalidate because project 9b8fad6dcc6540a0b01d3198fc9fc3ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:18:49.286794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:49.289258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a03d77e-60c7-44c8-9e0f-10274cd9b9f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:18:49.293117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1147/2329] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:18:48 2022] DELETE /identity/v3/projects/9b8fad6dcc6540a0b01d3198fc9fc3ef => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:49.307489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d5fb-3310-4442-a150-98afdf25a4ba None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:49.307900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d5fb-3310-4442-a150-98afdf25a4ba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:49.308276 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d5fb-3310-4442-a150-98afdf25a4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:49.330187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9774d5fb-3310-4442-a150-98afdf25a4ba None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:49.331678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9774d5fb-3310-4442-a150-98afdf25a4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:49.438477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1148/2330] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:49.442736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:49.444315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:49.444609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:49.455319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:49.456113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:49.456400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:49.456676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:49.468740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:49.486729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:49.504215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:49.505750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:49.523456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1149/2331] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:49.525686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:49.544114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a46db2-7795-4de0-ab8c-98120e5a6c13 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:49.560591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:49.561356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:49.561647 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:49.562732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:49.563690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:18:49.564093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:49.567230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:49.621603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:49.624162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:49.645797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1150/2332] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:18:49.650330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1183/2333] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:18:49.651597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:49.662611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0eee7-5703-45b5-bc12-743f396318fe admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:49.671735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:49.672429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:49.672717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:49.672999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:49.673724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:49.675735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:49.821478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1151/2334] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:49 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:49.847334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:49.886528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c3b03-15ab-4722-a374-a75e206324e3 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:50.046656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:50.047708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:50.048964 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:50.049631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:50.050581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:50.053387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:50.103555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1184/2335] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:49 2022] POST /identity/v3/users => generated 287 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:50.108442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:50.130568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659494e-737b-4a26-8c0e-82b222a0b433 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:50.142136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:50.142136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:50.142136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:50.154784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:50.154784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:50.156169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:50.177682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1152/2336] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:50.182342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:50.197419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70845583-82d3-4e4b-812e-3d04ee3d6197 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:50.205029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c32d-1802-4b8e-b5f4-7337c8b3402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:50.206487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c32d-1802-4b8e-b5f4-7337c8b3402f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:50.206979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c32d-1802-4b8e-b5f4-7337c8b3402f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:50.207293 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c32d-1802-4b8e-b5f4-7337c8b3402f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:50.209410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c32d-1802-4b8e-b5f4-7337c8b3402f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:50.211736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c32d-1802-4b8e-b5f4-7337c8b3402f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:50.228391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1185/2337] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:18:50.228784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:50.242344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fbd5a-6573-450f-92b9-feb9f5bc464b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:50.257127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:50.259826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:50.261066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:50.261588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] PATH_INFO: `/v3/projects/4a8a290c7de54fb892f408f3d4f1e657/users/516e13cf120a461d8632844453635fa9/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:50.267436 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:50.300536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8a290c7de54fb892f408f3d4f1e657, user_id=516e13cf120a461d8632844453635fa9, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:50.307513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:50.379853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1153/2338] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:50 2022] PUT /identity/v3/projects/4a8a290c7de54fb892f408f3d4f1e657/users/516e13cf120a461d8632844453635fa9/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:18:50.386977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9b620-2c08-4836-a9ea-2e7d1d204694 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:50.388461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9b620-2c08-4836-a9ea-2e7d1d204694 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:50.389465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9b620-2c08-4836-a9ea-2e7d1d204694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:50.434281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9ab9b620-2c08-4836-a9ea-2e7d1d204694 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:50.437030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9ab9b620-2c08-4836-a9ea-2e7d1d204694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:50.633133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1186/2339] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:18:50.659670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:50.662563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:50.852635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71770b-b6a5-4bc4-9ed3-fa4070e0ef55 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:50.861085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb6fb3-9f18-46c3-9794-b9eb05aeedf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:50.861765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6fb3-9f18-46c3-9794-b9eb05aeedf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:50.862046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6fb3-9f18-46c3-9794-b9eb05aeedf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:50.862320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6fb3-9f18-46c3-9794-b9eb05aeedf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:50.873171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb6fb3-9f18-46c3-9794-b9eb05aeedf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:50.874655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb6fb3-9f18-46c3-9794-b9eb05aeedf3 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:50.887790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1154/2340] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:51.664980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:51.665979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:51.666409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:51.705837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] MFA Rules not processed for user `d25025d4a28e451fb4e60b5a1e97ce4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:51.707520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:51.752664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:51.882004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1187/2341] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:53.051950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fe269-1d0d-4dbb-b821-444d0d403c74 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:53.052449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fe269-1d0d-4dbb-b821-444d0d403c74 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:53.052783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fe269-1d0d-4dbb-b821-444d0d403c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:53.164569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d05fe269-1d0d-4dbb-b821-444d0d403c74 None None] MFA Rules not processed for user `516e13cf120a461d8632844453635fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:53.167174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d05fe269-1d0d-4dbb-b821-444d0d403c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:53.440611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1155/2342] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:53.468937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:53.470388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:53.599846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c43105-b778-45bc-bf85-89a22e1bef77 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:53.612902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa791-e4e4-4988-ba3c-e698026c5340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:53.613761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa791-e4e4-4988-ba3c-e698026c5340 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:53.614245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa791-e4e4-4988-ba3c-e698026c5340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:53.614696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa791-e4e4-4988-ba3c-e698026c5340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:53.635634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa791-e4e4-4988-ba3c-e698026c5340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:53.637599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa791-e4e4-4988-ba3c-e698026c5340 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:53.669447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1188/2343] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:18:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:18:55.690454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:55.692014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:55.692502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:55.722619 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] MFA Rules not processed for user `516e13cf120a461d8632844453635fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:55.724309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:55.735138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1156/2344] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:56.063623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.064591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.065033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.121139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] MFA Rules not processed for user `23c7ecdb284d4ad0bda072a87ad8984c`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:18:56.123094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:56.223350 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1189/2345] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:18:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:18:56.227279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:56.228844 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:18:56.257533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8420c-f2a5-4bea-bc42-aa91755830ff None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:56.270846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:56.271980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.272370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.272739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.273556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:56.275806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:56.318793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1157/2346] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:18:56 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:18:56.323076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:56.338566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454103-3db7-4870-836d-a28570ae1eb4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:56.447405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:56.448278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.448637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.449109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.450968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:56.452414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:56.463375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:56.485235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1190/2347] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:18:56 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:18:56.489013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:56.512271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7460ff-0f72-4bb2-8de6-b7071fac0035 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:56.531664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:56.532379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.532669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.532999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.533754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:56.536838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:56.576355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1158/2348] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:56.577577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:56.591963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bf02e-eda5-4c12-90eb-921355679d94 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:56.598797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:56.600043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.600597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.602153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] PATH_INFO: `/v3/projects/ec8925e14c8245d8a8ea20df16b39f36/users/72de2072aa2e432380334a37e9061231/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.629166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8925e14c8245d8a8ea20df16b39f36, user_id=72de2072aa2e432380334a37e9061231, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:56.632516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:56.666848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1191/2349] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:56 2022] PUT /identity/v3/projects/ec8925e14c8245d8a8ea20df16b39f36/users/72de2072aa2e432380334a37e9061231/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:18:56.669589 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:56.701243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484ed5c-8eaf-4a41-b730-be314be4df97 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:56.792133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:18:56.859613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949bef0-adc2-40e7-8612-92b9350321d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:56.860367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949bef0-adc2-40e7-8612-92b9350321d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.860656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949bef0-adc2-40e7-8612-92b9350321d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.861500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949bef0-adc2-40e7-8612-92b9350321d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.862346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949bef0-adc2-40e7-8612-92b9350321d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:56.865225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949bef0-adc2-40e7-8612-92b9350321d0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:56.888858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1159/2350] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:18:56.889655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:18:56.911816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0079d-4a7c-40d1-9db6-d176f4fbc864 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:18:56.931887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:18:56.932803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:18:56.933087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:18:56.933361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] PATH_INFO: `/v3/projects/ec8925e14c8245d8a8ea20df16b39f36/users/72de2072aa2e432380334a37e9061231/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:18:56.949786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8925e14c8245d8a8ea20df16b39f36, user_id=72de2072aa2e432380334a37e9061231, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:18:56.963404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:18:57.006681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1192/2351] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:18:56 2022] PUT /identity/v3/projects/ec8925e14c8245d8a8ea20df16b39f36/users/72de2072aa2e432380334a37e9061231/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:19:06.875095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=12 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:06.993842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=3,keystone:SELECT=17 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:07.640495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbe012-7b75-4f7c-b315-bc3b2e6e7ad9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:07.640893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbe012-7b75-4f7c-b315-bc3b2e6e7ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:07.641435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbe012-7b75-4f7c-b315-bc3b2e6e7ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:07.676902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2bfbe012-7b75-4f7c-b315-bc3b2e6e7ad9 None None] MFA Rules not processed for user `72de2072aa2e432380334a37e9061231`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:07.686151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2bfbe012-7b75-4f7c-b315-bc3b2e6e7ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:07.919393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1160/2352] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:07.927394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:07.929258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:08.186321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e5617-8d2c-4c3a-ac3a-3759abfc4c11 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:08.192412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:08.193241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:08.193613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:08.193996 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:08.296505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:08.299261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:08.320574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1193/2353] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:09.854114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e01c-da22-4a3c-b775-dfe52471c0ba None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:09.856386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e01c-da22-4a3c-b775-dfe52471c0ba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:09.856757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e01c-da22-4a3c-b775-dfe52471c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:09.889355 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5472e01c-da22-4a3c-b775-dfe52471c0ba None None] MFA Rules not processed for user `23c7ecdb284d4ad0bda072a87ad8984c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:09.891088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5472e01c-da22-4a3c-b775-dfe52471c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:09.910038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1161/2354] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:15.243274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:15.245149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:15.340841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:15.357541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed52d4-6e8d-47d2-b422-17908e21c523 service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:15.364442 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:15.365113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:15.365391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:15.365665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:15.367794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:15.381198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:15.382927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea81eaf-39ab-4f19-aeb0-4cf848d4039d service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:15.394062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1194/2355] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:18.121046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd2bf-b28c-49c1-af00-61069c89ff8f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.121537 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd2bf-b28c-49c1-af00-61069c89ff8f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.121635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bd2bf-b28c-49c1-af00-61069c89ff8f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.123031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1162/2356] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:19:18.131444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.131917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.132142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.182439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:18.187873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:18.203916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1195/2357] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:18.293662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.293941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.294217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.348492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:18.349702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:18.420636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1163/2358] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:18.433954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:18.435706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:18.536163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae2c15-2d39-489f-8dad-5f2d1d68dd4d None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:18.541759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6d3b7-caf1-4ea5-83a8-60de75670aa3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:18.542424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6d3b7-caf1-4ea5-83a8-60de75670aa3 service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.542738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6d3b7-caf1-4ea5-83a8-60de75670aa3 service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.543085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6d3b7-caf1-4ea5-83a8-60de75670aa3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.551937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6d3b7-caf1-4ea5-83a8-60de75670aa3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:18.553582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6d3b7-caf1-4ea5-83a8-60de75670aa3 service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:18.562645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1196/2359] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:18.709021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:18.711157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:18.778818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:18.803614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfabbd-1334-477d-a2e1-822dd76ece30 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:18.807341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48c6b7-ac61-4df8-b9b1-054fbfc6b9c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:18.807992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48c6b7-ac61-4df8-b9b1-054fbfc6b9c2 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.808270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48c6b7-ac61-4df8-b9b1-054fbfc6b9c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.808541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48c6b7-ac61-4df8-b9b1-054fbfc6b9c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.818954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48c6b7-ac61-4df8-b9b1-054fbfc6b9c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:18.820412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48c6b7-ac61-4df8-b9b1-054fbfc6b9c2 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:18.831014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1164/2360] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:19:18.916331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb271c0-ea79-4a17-9871-d4263d12339e None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.916726 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb271c0-ea79-4a17-9871-d4263d12339e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.917004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb271c0-ea79-4a17-9871-d4263d12339e None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.918529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1197/2361] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:19:18.924493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e9516-92ab-421d-9d4d-04549ac64fe4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:18.924794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e9516-92ab-421d-9d4d-04549ac64fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:18.925155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e9516-92ab-421d-9d4d-04549ac64fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:18.946495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-204e9516-92ab-421d-9d4d-04549ac64fe4 None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:18.947854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-204e9516-92ab-421d-9d4d-04549ac64fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:19.005856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1165/2362] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:19:19.017782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb271c0-ea79-4a17-9871-d4263d12339e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:19.019433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8cb271c0-ea79-4a17-9871-d4263d12339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:19.033274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb271c0-ea79-4a17-9871-d4263d12339e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:19.036894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a7b25-52c9-463a-b19c-b6049955db18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:19.037541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a7b25-52c9-463a-b19c-b6049955db18 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.037822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a7b25-52c9-463a-b19c-b6049955db18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.038094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a7b25-52c9-463a-b19c-b6049955db18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.047312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a7b25-52c9-463a-b19c-b6049955db18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:19.048873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a7b25-52c9-463a-b19c-b6049955db18 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:19.058252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1198/2363] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:19.187011 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4903326-0b47-435a-8560-f9308d0b49d9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.187565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4903326-0b47-435a-8560-f9308d0b49d9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.188214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4903326-0b47-435a-8560-f9308d0b49d9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.190592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1166/2364] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:19:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:19:19.196902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc28197-7722-48d6-87b9-bd4a1db74bb0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.197480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc28197-7722-48d6-87b9-bd4a1db74bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.197812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc28197-7722-48d6-87b9-bd4a1db74bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.243050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-cbc28197-7722-48d6-87b9-bd4a1db74bb0 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:19.244279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cbc28197-7722-48d6-87b9-bd4a1db74bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:19.265212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1199/2365] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:19:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:19.283013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4903326-0b47-435a-8560-f9308d0b49d9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:19.297804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4903326-0b47-435a-8560-f9308d0b49d9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:19.303167 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:19.303851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.304213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.304485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.332356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:19.334808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:19.360999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1167/2366] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:19.406560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2107d-ca77-4934-bdcd-ac8dca21caae None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.406916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2107d-ca77-4934-bdcd-ac8dca21caae None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.407201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2107d-ca77-4934-bdcd-ac8dca21caae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.426094 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b5a2107d-ca77-4934-bdcd-ac8dca21caae None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:19.427349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b5a2107d-ca77-4934-bdcd-ac8dca21caae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:19.447736 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1200/2367] 10.222.0.64 () {58 vars in 967 bytes} [Fri Jun 24 10:19:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:19.455711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:19.468530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790bd10-e898-47f6-a393-0a6c232296b4 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:19.473715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:19.474448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.474751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.475396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.484267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:19.485882 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:19.499514 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1168/2368] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:19.732719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9392c55-0fb2-4c3e-acb3-d96d74bd35c5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.733004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9392c55-0fb2-4c3e-acb3-d96d74bd35c5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.733337 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9392c55-0fb2-4c3e-acb3-d96d74bd35c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.751630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b9392c55-0fb2-4c3e-acb3-d96d74bd35c5 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:19.752852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b9392c55-0fb2-4c3e-acb3-d96d74bd35c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:19.760176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1201/2369] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:19:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:19.773136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:19.788265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6fcfd-7fc2-4801-8b26-56e5cf9c4b23 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:19.793697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bccdd5-dece-4093-a1d5-055339f0d3e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:19.794367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bccdd5-dece-4093-a1d5-055339f0d3e6 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.794617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bccdd5-dece-4093-a1d5-055339f0d3e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.794943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bccdd5-dece-4093-a1d5-055339f0d3e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.804742 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bccdd5-dece-4093-a1d5-055339f0d3e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:19.806333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bccdd5-dece-4093-a1d5-055339f0d3e6 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:19.816754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1169/2370] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:19.852289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:19.852593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:19.852931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:19.874441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] MFA Rules not processed for user `516e13cf120a461d8632844453635fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:19.875945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:19.944117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1202/2371] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:20.214814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:20.215288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:20.215381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:20.242502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:20.243801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:20.253188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1170/2372] 10.222.0.64 () {58 vars in 967 bytes} [Fri Jun 24 10:19:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:20.272669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:20.279134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:20.286441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc622f-71b5-4079-920b-86f50a974c41 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:20.291690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bcfa0-c20e-4b98-b92c-ebad6c2d5cc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:20.292349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bcfa0-c20e-4b98-b92c-ebad6c2d5cc7 service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:20.292634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bcfa0-c20e-4b98-b92c-ebad6c2d5cc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:20.292913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bcfa0-c20e-4b98-b92c-ebad6c2d5cc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:20.302943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bcfa0-c20e-4b98-b92c-ebad6c2d5cc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:20.305514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bcfa0-c20e-4b98-b92c-ebad6c2d5cc7 service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:20.315555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1203/2373] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:22.652965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:22.654672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:22.735388 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb2083-de82-4326-9ab5-019629234725 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:22.740158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:22.741153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:22.741435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:22.741709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] PATH_INFO: `/v3/users/d25025d4a28e451fb4e60b5a1e97ce4c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:22.743026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25025d4a28e451fb4e60b5a1e97ce4c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:22.744997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:22.794490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:22.805520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25025d4a28e451fb4e60b5a1e97ce4c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:22.828666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25025d4a28e451fb4e60b5a1e97ce4c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:22.839018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25025d4a28e451fb4e60b5a1e97ce4c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:22.849297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25025d4a28e451fb4e60b5a1e97ce4c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:22.850106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25025d4a28e451fb4e60b5a1e97ce4c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:22.859221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1171/2374] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:19:22 2022] DELETE /identity/v3/users/d25025d4a28e451fb4e60b5a1e97ce4c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:19:22.864041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:22.864599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:22.864599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:22.883679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:22.885195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:22.961332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1204/2375] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:22.972772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:22.974331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:23.041654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73b690-cc2f-4ee5-b47d-3779f578cf80 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.045799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbbd41-5a63-4c4c-a628-662d5efe3359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.046509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbbd41-5a63-4c4c-a628-662d5efe3359 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.046830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbbd41-5a63-4c4c-a628-662d5efe3359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.047107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbbd41-5a63-4c4c-a628-662d5efe3359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.058645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbbd41-5a63-4c4c-a628-662d5efe3359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.060176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbbd41-5a63-4c4c-a628-662d5efe3359 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.071605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1172/2376] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:23.320095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.321654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:23.339146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fe008-629f-4902-90a1-43ff64a4e3f0 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.345558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.346676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.347155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.347695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] PATH_INFO: `/v3/projects/f68ef0ce863f43cdab4afcba5329f925` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.350224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68ef0ce863f43cdab4afcba5329f925)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.353349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.431515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68ef0ce863f43cdab4afcba5329f925', 'request_id': 'req-b37339f0-4746-4e37-8e99-9ab74a9995a9'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:23.438987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68ef0ce863f43cdab4afcba5329f925', 'request_id': 'req-b37339f0-4746-4e37-8e99-9ab74a9995a9'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:23.450204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68ef0ce863f43cdab4afcba5329f925', 'request_id': 'req-b37339f0-4746-4e37-8e99-9ab74a9995a9'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:23.465901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] The token cache is being invalidate because project f68ef0ce863f43cdab4afcba5329f925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:19:23.465901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:23.466584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-b37339f0-4746-4e37-8e99-9ab74a9995a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:23.469655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1205/2377] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:19:23 2022] DELETE /identity/v3/projects/f68ef0ce863f43cdab4afcba5329f925 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:19:23.481241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.481617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.481983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.503829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:23.505187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:23.569942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1173/2378] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:23.578283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.578283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.578643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.599721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:23.600934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:23.609197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1206/2379] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:23.613896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.624120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2827de9-bf67-4c42-a7f0-3c7f43dd736e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.628928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.629592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.629872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.630142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.630917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:19:23.631221 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.632893 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.649114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1174/2380] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:19:23.650931 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.660642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c226e-9607-4c87-af77-457c59c52ee2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.665632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.666388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.666670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.666974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.667693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.669624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.702793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1207/2381] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:19:23 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:19:23.704211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.714837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5d81-15b3-47db-9b90-b9184e7cf075 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.762732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:23.778140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.778874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.779161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.779444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.780144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.782278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.803397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1175/2382] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:19:23 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:19:23.807586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.818067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6af43-50d4-406d-9e85-1a995f4ecfe1 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.822899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.823646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.823997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.824338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.825162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.826896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.838245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1208/2383] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:19:23.843061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.854153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fb839-9246-4696-9789-33b02bb974c1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.859970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d60c51-0b90-46e4-a019-82713020267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.860649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60c51-0b90-46e4-a019-82713020267b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.860934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60c51-0b90-46e4-a019-82713020267b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.861208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60c51-0b90-46e4-a019-82713020267b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.861951 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d60c51-0b90-46e4-a019-82713020267b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.863637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d60c51-0b90-46e4-a019-82713020267b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.874527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1176/2384] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:19:23.879338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:23.891454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a3f34-d915-4b46-a389-95a97e705518 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:23.896536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:23.897500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.897751 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.898028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] PATH_INFO: `/v3/projects/50556a2b3c4741c8b660428549ba4b39/users/f67c664325d645c78dab4e77b51d739c/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.914545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50556a2b3c4741c8b660428549ba4b39, user_id=f67c664325d645c78dab4e77b51d739c, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:23.917294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:23.931107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1209/2385] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:19:23 2022] PUT /identity/v3/projects/50556a2b3c4741c8b660428549ba4b39/users/f67c664325d645c78dab4e77b51d739c/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:19:23.941434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf16e7-8265-417a-a694-c4b03f133166 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:23.941714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf16e7-8265-417a-a694-c4b03f133166 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:23.941993 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf16e7-8265-417a-a694-c4b03f133166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:23.970952 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b5cf16e7-8265-417a-a694-c4b03f133166 None None] MFA Rules not processed for user `f67c664325d645c78dab4e77b51d739c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:23.972186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b5cf16e7-8265-417a-a694-c4b03f133166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:24.042565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1177/2386] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:24.058502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:24.060415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:24.073954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:24.154679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fea09-227b-4b2f-a48d-c2de909b4cb7 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:24.158626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:24.159338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:24.159623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:24.159910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:24.169290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:24.170762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:24.183787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1210/2387] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:24.474541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:24.474872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:24.475240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:24.496891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:24.498099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:24.608864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1178/2388] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:19:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:24.623752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:24.625627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:24.718179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689545f-cb6c-40f8-8222-1289fd909a39 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:24.729523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad02d9-6e0f-4975-93ce-f0ea0ee1f37f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:24.730167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad02d9-6e0f-4975-93ce-f0ea0ee1f37f service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:24.730579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad02d9-6e0f-4975-93ce-f0ea0ee1f37f service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:24.730750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad02d9-6e0f-4975-93ce-f0ea0ee1f37f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:24.740980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad02d9-6e0f-4975-93ce-f0ea0ee1f37f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:24.742617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad02d9-6e0f-4975-93ce-f0ea0ee1f37f service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:24.755460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1211/2389] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:34.585166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:34.751178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=63 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:44.274603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:44.345389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ca964-4ac6-4fc0-910e-0c3b97326d7b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:44.353111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:44.354363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:44.354813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:44.355515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] PATH_INFO: `/v3/users/f67c664325d645c78dab4e77b51d739c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:44.357714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67c664325d645c78dab4e77b51d739c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:44.360643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:44.431175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67c664325d645c78dab4e77b51d739c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:44.448591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67c664325d645c78dab4e77b51d739c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:44.457712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67c664325d645c78dab4e77b51d739c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:44.465669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67c664325d645c78dab4e77b51d739c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:44.466306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67c664325d645c78dab4e77b51d739c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:44.472573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1179/2390] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:19:44 2022] DELETE /identity/v3/users/f67c664325d645c78dab4e77b51d739c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:19:44.477507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:44.477998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:44.478080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:44.495738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:44.497025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:44.563050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1212/2391] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:44.573740 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:44.575293 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:44.637181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63039fa2-777e-4b90-99d1-ab9f08ca6630 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:44.641160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef8f5-bd07-4082-a95d-d6bb6bea4907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:44.641801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef8f5-bd07-4082-a95d-d6bb6bea4907 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:44.642082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef8f5-bd07-4082-a95d-d6bb6bea4907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:44.642348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef8f5-bd07-4082-a95d-d6bb6bea4907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:44.652909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef8f5-bd07-4082-a95d-d6bb6bea4907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:44.654523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef8f5-bd07-4082-a95d-d6bb6bea4907 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:44.665297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1180/2392] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:19:44.973770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:44.975394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:44.989623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4101-f46a-4382-b894-86e383082110 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:44.996380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:44.997534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:44.998166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:44.998600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] PATH_INFO: `/v3/projects/50556a2b3c4741c8b660428549ba4b39` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:45.000561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50556a2b3c4741c8b660428549ba4b39)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:45.002690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:45.064218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50556a2b3c4741c8b660428549ba4b39', 'request_id': 'req-8c2a319b-fc3e-4281-934f-f3bbf620a29c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:45.071072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50556a2b3c4741c8b660428549ba4b39', 'request_id': 'req-8c2a319b-fc3e-4281-934f-f3bbf620a29c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:45.078913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50556a2b3c4741c8b660428549ba4b39', 'request_id': 'req-8c2a319b-fc3e-4281-934f-f3bbf620a29c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:45.090778 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] The token cache is being invalidate because project 50556a2b3c4741c8b660428549ba4b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:19:45.091084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:45.091761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-8c2a319b-fc3e-4281-934f-f3bbf620a29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:19:45.093341 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1213/2393] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:19:44 2022] DELETE /identity/v3/projects/50556a2b3c4741c8b660428549ba4b39 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:19:49.467708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:49.468212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:49.468212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:49.486971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] MFA Rules not processed for user `acbafaab99f848828c304ca5a45a55dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:49.488197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:49.507015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:49.548541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1181/2394] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:19:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:52.534861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5a93-8d19-4622-b4b8-202d53751ef6 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:52.536743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5a93-8d19-4622-b4b8-202d53751ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:52.537261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5a93-8d19-4622-b4b8-202d53751ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:52.558732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1e7b5a93-8d19-4622-b4b8-202d53751ef6 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:52.559999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1e7b5a93-8d19-4622-b4b8-202d53751ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:52.620968 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1214/2395] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:19:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:19:52.643835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:52.645347 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:52.735335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf389057-180b-45b5-a101-29a299dbb4ee None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:52.745140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:52.746288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:52.746680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:52.747131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:52.759299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:52.760965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:52.769474 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1182/2396] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:54.907422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:54.908033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:54.908254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:54.932587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:19:54.934861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:54.955025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1215/2397] 10.222.0.64 () {58 vars in 967 bytes} [Fri Jun 24 10:19:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:19:54.977493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:55.000605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93112431-8895-4b88-a21f-3cdb7fc9a405 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:55.006666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f1389-2398-47fb-a1bd-4fc6ef4cb051 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:55.007478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f1389-2398-47fb-a1bd-4fc6ef4cb051 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:55.007682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f1389-2398-47fb-a1bd-4fc6ef4cb051 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:55.007883 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f1389-2398-47fb-a1bd-4fc6ef4cb051 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:55.018618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f1389-2398-47fb-a1bd-4fc6ef4cb051 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:55.020377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f1389-2398-47fb-a1bd-4fc6ef4cb051 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:55.032232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1183/2398] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:19:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:19:59.050234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:19:59.052433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:59.086321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:19:59.127042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dae776-1f89-4dc1-9b5a-2400da38e7e8 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:19:59.131954 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:19:59.132626 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:19:59.132906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:19:59.133180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:19:59.135157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:19:59.210724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:19:59.212347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:19:59.227865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1216/2399] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:19:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:20:05.026267 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=53 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:09.185672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:09.186258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:09.186258 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:09.205622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:09.206877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:09.216861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1184/2400] 10.222.0.64 () {58 vars in 967 bytes} [Fri Jun 24 10:20:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:09.223129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=40 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:09.230815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:09.243025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80641ea2-a35b-4a6c-b1c9-7907ecc79756 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:09.248886 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:09.249534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:09.249818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:09.250093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:09.258679 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:09.260370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:09.268847 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1217/2401] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:20:11.262166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:11.263758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:11.337159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229a376-ed92-4f9d-a92d-77090a049b2a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:11.342179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:11.343085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:11.343363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:11.343631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] PATH_INFO: `/v3/users/516e13cf120a461d8632844453635fa9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:11.344824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516e13cf120a461d8632844453635fa9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:11.346791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:11.401048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516e13cf120a461d8632844453635fa9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.413247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516e13cf120a461d8632844453635fa9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.420901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516e13cf120a461d8632844453635fa9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.427922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516e13cf120a461d8632844453635fa9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.428568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-daf53bf1-73bc-48ea-9ce2-a43b8d2aca99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516e13cf120a461d8632844453635fa9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.434721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1185/2402] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:20:11 2022] DELETE /identity/v3/users/516e13cf120a461d8632844453635fa9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:20:11.635604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:11.646673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf34e2-4a2f-49d1-bce1-a9907c8943da service swift] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:11.726735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:11.727769 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:11.728122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:11.728468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] PATH_INFO: `/v3/projects/4a8a290c7de54fb892f408f3d4f1e657` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:11.729922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8a290c7de54fb892f408f3d4f1e657)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:11.732257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:11.783857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8a290c7de54fb892f408f3d4f1e657', 'request_id': 'req-43fcd4d2-5a5d-40ec-b850-806bfd386120'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.790653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8a290c7de54fb892f408f3d4f1e657', 'request_id': 'req-43fcd4d2-5a5d-40ec-b850-806bfd386120'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.800628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8a290c7de54fb892f408f3d4f1e657', 'request_id': 'req-43fcd4d2-5a5d-40ec-b850-806bfd386120'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.812793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] The token cache is being invalidate because project 4a8a290c7de54fb892f408f3d4f1e657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:20:11.813087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.813744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-43fcd4d2-5a5d-40ec-b850-806bfd386120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:11.815559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1218/2403] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:20:11 2022] DELETE /identity/v3/projects/4a8a290c7de54fb892f408f3d4f1e657 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:20:11.821016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d143-808e-4957-be79-e03be33cd37e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:11.821312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d143-808e-4957-be79-e03be33cd37e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:11.821595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d143-808e-4957-be79-e03be33cd37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:11.842380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7756d143-808e-4957-be79-e03be33cd37e None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:11.843644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7756d143-808e-4957-be79-e03be33cd37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:11.904805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1186/2404] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:11.912412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:11.912707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:11.912988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:11.932881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:11.934146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:11.942948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1219/2405] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:11.947572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d143-808e-4957-be79-e03be33cd37e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:11.958973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d143-808e-4957-be79-e03be33cd37e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:11.965232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:11.966189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:11.966551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:11.966859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:11.967902 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:20:11.968263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:11.969888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:11.988425 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:11.989945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1187/2406] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:20:11.993295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:12.003330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d59ec-4723-41c4-8bc3-73f5ec3340fb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:12.008267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:12.008925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.009206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.009502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.010346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:12.012310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:12.043689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1220/2407] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:20:11 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:20:12.047837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:12.058333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0866dc-9380-4de0-b7be-5f72bedc84ec admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:12.119390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:12.120054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.120338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.120610 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.121329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:12.123467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:12.147559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1188/2408] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:20:12 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:20:12.152786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:12.164098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a757c-d770-4c0e-85e5-3acd744c0e14 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:12.169617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:12.170298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.170579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.170967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.171705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:12.173332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:12.185135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1221/2409] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:20:12.189279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:12.200178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d3054-6369-45d5-95ec-b39559fab45a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:12.205744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496702d-be59-43ff-b0c1-fda80938a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:12.206422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496702d-be59-43ff-b0c1-fda80938a524 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.206719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496702d-be59-43ff-b0c1-fda80938a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.207015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496702d-be59-43ff-b0c1-fda80938a524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.207741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496702d-be59-43ff-b0c1-fda80938a524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:12.209354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496702d-be59-43ff-b0c1-fda80938a524 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:12.220466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1189/2410] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:20:12.225041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:12.235783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1589d-6ee4-4577-add8-06520baaa36e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:12.240752 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:12.241634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.241909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.242179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] PATH_INFO: `/v3/projects/c947b446e8cc4a4fa9c4438d048b7d1e/users/ad1ba2b7689d449eb76d86af3c9a2956/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.257952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c947b446e8cc4a4fa9c4438d048b7d1e, user_id=ad1ba2b7689d449eb76d86af3c9a2956, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:12.260675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:12.275747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1222/2411] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:20:12 2022] PUT /identity/v3/projects/c947b446e8cc4a4fa9c4438d048b7d1e/users/ad1ba2b7689d449eb76d86af3c9a2956/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:20:12.285898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5e02-68e0-4683-97a9-3dea882d7dc7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.286373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5e02-68e0-4683-97a9-3dea882d7dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.286511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5e02-68e0-4683-97a9-3dea882d7dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.316975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-826f5e02-68e0-4683-97a9-3dea882d7dc7 None None] MFA Rules not processed for user `ad1ba2b7689d449eb76d86af3c9a2956`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:12.318188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-826f5e02-68e0-4683-97a9-3dea882d7dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:12.387971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1190/2412] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:12.400043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:12.402043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:12.418556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:12.467923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b64ed4-8029-4cff-86cc-b18aab474ee9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:12.472212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:12.472869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:12.473153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:12.473434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:12.482406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:12.483902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:12.492989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1223/2413] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:20:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:20:22.390290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:22.489032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=20 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:46.957407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed9da-b61e-4e8a-a4fe-2b8fe404d88f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:46.957961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed9da-b61e-4e8a-a4fe-2b8fe404d88f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:46.958071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed9da-b61e-4e8a-a4fe-2b8fe404d88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:46.976869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-086ed9da-b61e-4e8a-a4fe-2b8fe404d88f None None] MFA Rules not processed for user `0a66b13572904a309d3180afc7278070`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:46.978094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-086ed9da-b61e-4e8a-a4fe-2b8fe404d88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:47.044802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1191/2414] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:50.043436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:50.114491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d1dfa-aeb8-4b0f-95e3-4968714e3d19 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:50.119414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:50.120304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:50.120577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:50.120843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] PATH_INFO: `/v3/users/ad1ba2b7689d449eb76d86af3c9a2956` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:50.122018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1ba2b7689d449eb76d86af3c9a2956)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:50.124038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:50.176099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1ba2b7689d449eb76d86af3c9a2956'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.189550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1ba2b7689d449eb76d86af3c9a2956'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.198731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1ba2b7689d449eb76d86af3c9a2956'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.206536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1ba2b7689d449eb76d86af3c9a2956'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.207237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1ba2b7689d449eb76d86af3c9a2956'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.213434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1224/2415] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:20:50 2022] DELETE /identity/v3/users/ad1ba2b7689d449eb76d86af3c9a2956 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:20:50.218799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:50.219301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:50.219476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:50.239101 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:50.240228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:50.305491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1192/2416] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:50.316149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:50.317789 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:50.380041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a0453-45d4-461e-b239-f5eb8743bb7e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:50.384413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7fb96-1668-4185-9bb9-ed9d6761e6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:50.385196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7fb96-1668-4185-9bb9-ed9d6761e6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:50.385479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7fb96-1668-4185-9bb9-ed9d6761e6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:50.385827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7fb96-1668-4185-9bb9-ed9d6761e6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:50.397509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7fb96-1668-4185-9bb9-ed9d6761e6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:50.399267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7fb96-1668-4185-9bb9-ed9d6761e6a8 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:50.409407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1225/2417] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:20:50.760601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:50.762818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:50.779793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbb4a0-0a77-46af-bf84-d8f45c607cca None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:50.786257 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:50.787683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:50.788150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:50.788593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] PATH_INFO: `/v3/projects/c947b446e8cc4a4fa9c4438d048b7d1e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:50.790118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c947b446e8cc4a4fa9c4438d048b7d1e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:50.792363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:50.855231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c947b446e8cc4a4fa9c4438d048b7d1e', 'request_id': 'req-9bc4c6d4-2988-47da-8c88-089447b75342'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.865170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c947b446e8cc4a4fa9c4438d048b7d1e', 'request_id': 'req-9bc4c6d4-2988-47da-8c88-089447b75342'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.873346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c947b446e8cc4a4fa9c4438d048b7d1e', 'request_id': 'req-9bc4c6d4-2988-47da-8c88-089447b75342'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.890226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] The token cache is being invalidate because project c947b446e8cc4a4fa9c4438d048b7d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:20:50.890963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.891912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9bc4c6d4-2988-47da-8c88-089447b75342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:20:50.894630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1193/2418] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:20:50 2022] DELETE /identity/v3/projects/c947b446e8cc4a4fa9c4438d048b7d1e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:20:50.901219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:50.902212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:50.902748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:50.924212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:50.925842 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:50.957702 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:51.022181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1226/2419] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:51.029960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.030395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.030954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.056075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:51.058737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:51.070176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1194/2420] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:51.074250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.085067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e456-063c-439e-b49a-4c2ffa03db88 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.090126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.090843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.091131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.091405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.092189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:20:51.092521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.094161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.109200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1227/2421] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:20:51.114432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.126555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cf946-25be-4012-9a26-072df39b13a4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.134207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.134207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.134207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.134593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.135453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.137430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.160596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:51.168656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1195/2422] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:20:51 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:20:51.173435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.184088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa3d8-72a5-4298-87a0-d977294139d5 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.245451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.246064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.246351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.246624 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.247889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.249312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.275314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1228/2423] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:20:51 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:20:51.276548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.288864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2742e3-1758-4b02-8bf0-827578f29b27 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.294845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.295543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.295800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.296068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.296813 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.298447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.312581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1196/2424] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:20:51.318023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.328594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee46b4-2e8e-4e71-b54d-b4de4956bf67 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.335102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c5e3-c799-4cfd-9fb6-84f596987d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.335812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c5e3-c799-4cfd-9fb6-84f596987d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.336089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c5e3-c799-4cfd-9fb6-84f596987d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.336413 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c5e3-c799-4cfd-9fb6-84f596987d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.337149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c5e3-c799-4cfd-9fb6-84f596987d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.339208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c5e3-c799-4cfd-9fb6-84f596987d49 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.350656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1229/2425] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:20:51.355435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.367598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fe360-6fce-46bd-81d1-df3f40e48915 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.372757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.373653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.373927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.374193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] PATH_INFO: `/v3/projects/34f04f4ab2294464bdc83acecb4d9a79/users/ef90ff4734c249ab965ebf920270b20b/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.392350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f04f4ab2294464bdc83acecb4d9a79, user_id=ef90ff4734c249ab965ebf920270b20b, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.395708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.412327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1197/2426] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:20:51 2022] PUT /identity/v3/projects/34f04f4ab2294464bdc83acecb4d9a79/users/ef90ff4734c249ab965ebf920270b20b/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:20:51.418020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd797-5119-460b-811a-6e218870f874 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.418304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd797-5119-460b-811a-6e218870f874 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.418608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd797-5119-460b-811a-6e218870f874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.437705 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d9fbd797-5119-460b-811a-6e218870f874 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:51.438980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d9fbd797-5119-460b-811a-6e218870f874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:51.516800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1230/2427] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:51.531959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:51.533892 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:51.607850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afd0e0-c798-4eab-89b7-656604694906 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:51.612240 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:51.613247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:51.613247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:51.613462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:51.623868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:51.625614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:51.635911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1198/2428] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:20:57.228738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.229336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.229568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.264432 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] MFA Rules not processed for user `ef90ff4734c249ab965ebf920270b20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:20:57.266093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:20:57.291119 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:20:57.341252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1231/2429] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:20:57.345477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:57.359625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff8e3b-7728-4bd3-93da-2777da283d83 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:57.365381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:57.366045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.366323 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.368678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.369573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:57.371585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:57.403013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1199/2430] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:20:57 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:20:57.406928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:57.419899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b791e-1afe-4a82-93dd-6a94be42cf0a None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:57.485688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:57.487416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.487746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.488212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.488980 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:57.491161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:57.535246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:57.549781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1232/2431] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:20:57 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:20:57.550935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcc488-3c11-41ce-91bd-06ed89e44fda admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:57.567766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:57.568604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.568873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.569152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.569977 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:57.571785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:57.584195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1200/2432] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:20:57.590267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:57.611771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220a0f-578e-483f-8e3c-3df20ae063b5 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:57.617539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:57.618665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.619107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.619482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] PATH_INFO: `/v3/projects/05a745d261eb464a95f6f1437e87ad8d/users/ace547a802324ef99b14ae40b1fe1370/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.640813 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a745d261eb464a95f6f1437e87ad8d, user_id=ace547a802324ef99b14ae40b1fe1370, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:57.644566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:57.664643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1233/2433] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:20:57 2022] PUT /identity/v3/projects/05a745d261eb464a95f6f1437e87ad8d/users/ace547a802324ef99b14ae40b1fe1370/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:20:57.668288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:57.680421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd3434-94d8-4f93-a65e-13e0266373eb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:57.771346 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82000aaa-126c-4aa9-a836-6090f0b012b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:57.772075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000aaa-126c-4aa9-a836-6090f0b012b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.772392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000aaa-126c-4aa9-a836-6090f0b012b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.772668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000aaa-126c-4aa9-a836-6090f0b012b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.773728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82000aaa-126c-4aa9-a836-6090f0b012b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:57.775698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82000aaa-126c-4aa9-a836-6090f0b012b2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:57.788989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1201/2434] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:20:57.793054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:20:57.805667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d5ca-d964-49fa-8a88-9129be9e2603 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:20:57.811375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:20:57.812373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:20:57.812694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:20:57.812976 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] PATH_INFO: `/v3/projects/05a745d261eb464a95f6f1437e87ad8d/users/ace547a802324ef99b14ae40b1fe1370/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:20:57.815242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a745d261eb464a95f6f1437e87ad8d, user_id=ace547a802324ef99b14ae40b1fe1370, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:20:57.818391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:20:57.832297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1234/2435] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:20:57 2022] PUT /identity/v3/projects/05a745d261eb464a95f6f1437e87ad8d/users/ace547a802324ef99b14ae40b1fe1370/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:04.035151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f344b5-d00a-4cc7-94cf-9797cdb9ddce None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:04.035701 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f344b5-d00a-4cc7-94cf-9797cdb9ddce None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:04.036086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f344b5-d00a-4cc7-94cf-9797cdb9ddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:04.080449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-94f344b5-d00a-4cc7-94cf-9797cdb9ddce None None] MFA Rules not processed for user `ace547a802324ef99b14ae40b1fe1370`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:04.082364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-94f344b5-d00a-4cc7-94cf-9797cdb9ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:04.112541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:04.183924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1202/2436] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:04.196521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:04.198498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:04.271935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732beb-f7c8-4669-8cf3-094a7d703067 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:04.276314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:04.276991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:04.277278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:04.277558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:04.306031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:04.342020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:04.343523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:04.354021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1235/2437] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:04.487177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:04.487670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:04.488078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:04.519556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:04.521470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:04.654860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1203/2438] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:21:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:04.688648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:04.693716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:04.806783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386caa4-4807-4380-90bc-3e00e962e728 service glance] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:04.812371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3b0a-f2e0-47b5-a7cf-150b8201a351 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:04.813118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3b0a-f2e0-47b5-a7cf-150b8201a351 service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:04.813470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3b0a-f2e0-47b5-a7cf-150b8201a351 service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:04.813814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3b0a-f2e0-47b5-a7cf-150b8201a351 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:04.824422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff3b0a-f2e0-47b5-a7cf-150b8201a351 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:04.826160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff3b0a-f2e0-47b5-a7cf-150b8201a351 service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:04.851444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1236/2439] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:21:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:21:08.882583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:08.986484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1113f3-2017-45c8-92d3-c60b06e998b0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:08.991218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:08.991891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:08.992175 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:08.992613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:08.994503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:09.134622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:09.136133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:09.178544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1204/2440] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:14.735282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:14.735282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:14.735282 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:14.776280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] MFA Rules not processed for user `6ca14b36fc754fe9b24abdfecf0f64c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:14.779096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:14.859056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1237/2441] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:15.398601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:15.400690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:15.413349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3ade5-e639-47ba-b233-9ea665ae0987 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:15.419192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:15.420324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:15.420799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:15.421262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] PATH_INFO: `/v3/users/acbafaab99f848828c304ca5a45a55dc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:15.423179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acbafaab99f848828c304ca5a45a55dc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:15.426079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:15.432299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:15.511255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acbafaab99f848828c304ca5a45a55dc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:15.534857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acbafaab99f848828c304ca5a45a55dc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:15.561682 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acbafaab99f848828c304ca5a45a55dc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:15.570255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acbafaab99f848828c304ca5a45a55dc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:15.571795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0eb94f59-e487-4221-8ab2-1fd0d73f6802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acbafaab99f848828c304ca5a45a55dc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:15.586801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1205/2442] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:15 2022] DELETE /identity/v3/users/acbafaab99f848828c304ca5a45a55dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:15.863865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:15.883660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257c07-678b-4312-941f-de9bc99a6970 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.024306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.025261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.025551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.025832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] PATH_INFO: `/v3/projects/172381cd65a94b1fa2939d6af673f6ba` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.027099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172381cd65a94b1fa2939d6af673f6ba)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.032661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.034740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:16.108265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172381cd65a94b1fa2939d6af673f6ba', 'request_id': 'req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:16.116578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172381cd65a94b1fa2939d6af673f6ba', 'request_id': 'req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:16.127333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172381cd65a94b1fa2939d6af673f6ba', 'request_id': 'req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:16.151542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] The token cache is being invalidate because project 172381cd65a94b1fa2939d6af673f6ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:16.151984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:16.153819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-ec60a2a7-d26d-40ba-8b6f-dc65829d7898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:16.155903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1238/2443] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:15 2022] DELETE /identity/v3/projects/172381cd65a94b1fa2939d6af673f6ba => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:16.161478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.161764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.162041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.185876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:16.187659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:16.277978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1206/2444] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:16.280138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.280464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.280703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.305843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:16.307209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:16.316766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1239/2445] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:16.321407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.333765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b35c2-1f64-41e6-b17c-7e58eb0eb1a3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.340086 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.341066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.341508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.341931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.342995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:16.343512 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.345673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.363254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1207/2446] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:16.368035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.381665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191680f-db18-4b3d-8938-f2ff4e31dffa None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.388787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.389770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.390064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.390342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.391113 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.393366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.430030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1240/2447] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:16 2022] POST /identity/v3/projects => generated 363 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:16.434780 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.448563 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd8c0-c205-4520-acc6-ab116e86bf2a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.537728 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.538800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.539315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.539685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.540540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.542479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.570062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:16.580814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1208/2448] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:16 2022] POST /identity/v3/users => generated 282 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:16.583318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.606534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd11a3-bfdb-4664-9cd1-d442364c6e0e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.615568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.616873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.617436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.617903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.619022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.621783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.636959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1241/2449] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:16.642504 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.655354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c921d-fab7-4f06-b428-d13e59cc2a0c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.662420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598c14b-7671-45ad-acdf-f6c3ec06bf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.663169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598c14b-7671-45ad-acdf-f6c3ec06bf97 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.663670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598c14b-7671-45ad-acdf-f6c3ec06bf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.663949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598c14b-7671-45ad-acdf-f6c3ec06bf97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.665123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598c14b-7671-45ad-acdf-f6c3ec06bf97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.668422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598c14b-7671-45ad-acdf-f6c3ec06bf97 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.685357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1209/2450] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:16.695241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.709407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7e6d-4e5d-48b9-8b2c-b56cd97eac0a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.717154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.718103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.718385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.718657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] PATH_INFO: `/v3/projects/3a46e63cdb7b4c4487115ae014cd80bf/users/7a75e1dc62bb49d29e85a7b715e077e4/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.739299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a46e63cdb7b4c4487115ae014cd80bf, user_id=7a75e1dc62bb49d29e85a7b715e077e4, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.742221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:16.760023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1242/2451] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:16 2022] PUT /identity/v3/projects/3a46e63cdb7b4c4487115ae014cd80bf/users/7a75e1dc62bb49d29e85a7b715e077e4/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:16.772472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.772865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.773244 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.805833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] MFA Rules not processed for user `7a75e1dc62bb49d29e85a7b715e077e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:16.807226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:16.886264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1210/2452] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:16.888195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:16.973858 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d08d-2159-49f2-a132-131139e8d507 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:16.983868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:16.985666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:16.985666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:16.985666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:16.985666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:16.988242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.050354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1243/2453] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:16 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:17.054465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:17.071485 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1651be-22c7-4b79-b143-de94fd521843 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:17.272871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:17.273656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.274051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.274334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.275309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:17.281354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.319517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1211/2454] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:17 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:17.323414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:17.342481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5e4f2-4459-4dac-a3eb-f9af3e355809 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:17.356720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:17.357784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.358957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.360110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.361223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:17.363151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.369945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:17.379737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1244/2455] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:17.383495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:17.404071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfecb2-9768-4b65-88a5-acb76f06a1e6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:17.418535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:17.419506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.419792 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.420069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] PATH_INFO: `/v3/projects/a5a1c2bc36f94f4ca140be9fbcb03d06/users/0e604eee245e415a8234a02438d1efc5/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.439273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a1c2bc36f94f4ca140be9fbcb03d06, user_id=0e604eee245e415a8234a02438d1efc5, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:17.442074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.464040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1212/2456] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:17 2022] PUT /identity/v3/projects/a5a1c2bc36f94f4ca140be9fbcb03d06/users/0e604eee245e415a8234a02438d1efc5/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:17.465077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:17.476999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ca6d6-fb7a-4fa5-bdc2-d12de6719b29 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:17.577360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd780299-c722-4627-9da3-7e3ce1ab4653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:17.578001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780299-c722-4627-9da3-7e3ce1ab4653 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.578290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780299-c722-4627-9da3-7e3ce1ab4653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.578568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780299-c722-4627-9da3-7e3ce1ab4653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.580300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780299-c722-4627-9da3-7e3ce1ab4653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:17.582152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780299-c722-4627-9da3-7e3ce1ab4653 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.594081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1245/2457] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:17.598356 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:17.615525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e600b-3795-4a0c-bd0b-4e4029cd5488 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:17.621623 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:17.622963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.623259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.623534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] PATH_INFO: `/v3/projects/a5a1c2bc36f94f4ca140be9fbcb03d06/users/0e604eee245e415a8234a02438d1efc5/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.625861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a1c2bc36f94f4ca140be9fbcb03d06, user_id=0e604eee245e415a8234a02438d1efc5, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:17.630787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.641879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1213/2458] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:17 2022] PUT /identity/v3/projects/a5a1c2bc36f94f4ca140be9fbcb03d06/users/0e604eee245e415a8234a02438d1efc5/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:21:17.654737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.655610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.655961 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.688353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] MFA Rules not processed for user `0e604eee245e415a8234a02438d1efc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:17.690277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:17.793255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1246/2459] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:17.809080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:17.812551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:17.866139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:17.923842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd646cc1-df54-4315-bd91-ebfbe3c005d6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:17.936371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2be76-7cb8-4851-b0b9-b260c2f82edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:17.936675 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2be76-7cb8-4851-b0b9-b260c2f82edd service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:17.937023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2be76-7cb8-4851-b0b9-b260c2f82edd service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:17.943689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2be76-7cb8-4851-b0b9-b260c2f82edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:17.959760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2be76-7cb8-4851-b0b9-b260c2f82edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:17.961400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2be76-7cb8-4851-b0b9-b260c2f82edd service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:17.975614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1214/2460] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:18.555660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:18.660321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42da6a7-53b6-491a-82f0-9feb4b63a9dc None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:18.666362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:18.667287 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:18.667568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:18.667843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] PATH_INFO: `/v3/users/7a75e1dc62bb49d29e85a7b715e077e4` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:18.669054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a75e1dc62bb49d29e85a7b715e077e4)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:18.671047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:18.711607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:18.745368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a75e1dc62bb49d29e85a7b715e077e4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:18.761149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a75e1dc62bb49d29e85a7b715e077e4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:18.775294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a75e1dc62bb49d29e85a7b715e077e4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:18.784772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a75e1dc62bb49d29e85a7b715e077e4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:18.785673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c3b222e6-92ea-4297-972f-6217fc87452c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a75e1dc62bb49d29e85a7b715e077e4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:18.793014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1247/2461] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:18 2022] DELETE /identity/v3/users/7a75e1dc62bb49d29e85a7b715e077e4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:18.797331 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a08302-8065-403b-b63b-578909d38433 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:18.798658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a08302-8065-403b-b63b-578909d38433 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:18.799760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a08302-8065-403b-b63b-578909d38433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:18.837005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-90a08302-8065-403b-b63b-578909d38433 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:18.839036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-90a08302-8065-403b-b63b-578909d38433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:18.852151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:18.852419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:18.852700 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:18.880651 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] MFA Rules not processed for user `ef90ff4734c249ab965ebf920270b20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:18.883107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:18.923635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1215/2462] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:18.933704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a08302-8065-403b-b63b-578909d38433 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:18.935162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-90a08302-8065-403b-b63b-578909d38433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:18.969994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1248/2463] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:19.002226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a08302-8065-403b-b63b-578909d38433 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:19.005962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:19.006626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:19.006942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:19.007219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:19.017637 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:19.019243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:19.034897 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1216/2464] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:19.655152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:19.661502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:19.701162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca157-3e05-4c50-926a-5475cc8f974f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:19.720126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:19.721729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:19.721729 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:19.721909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] PATH_INFO: `/v3/projects/3a46e63cdb7b4c4487115ae014cd80bf` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:19.724189 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a46e63cdb7b4c4487115ae014cd80bf)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:19.726227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:19.790376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a46e63cdb7b4c4487115ae014cd80bf', 'request_id': 'req-17613957-8c78-4829-afb2-b6d6eda8dc20'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.797532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a46e63cdb7b4c4487115ae014cd80bf', 'request_id': 'req-17613957-8c78-4829-afb2-b6d6eda8dc20'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.807247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a46e63cdb7b4c4487115ae014cd80bf', 'request_id': 'req-17613957-8c78-4829-afb2-b6d6eda8dc20'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.819170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] The token cache is being invalidate because project 3a46e63cdb7b4c4487115ae014cd80bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:19.819465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.820125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.822093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1249/2465] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:19 2022] DELETE /identity/v3/projects/3a46e63cdb7b4c4487115ae014cd80bf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:19.825719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:19.836372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12961e89-086f-4bc1-aaee-c6ff438c6450 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:19.898696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:19.899622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:19.899907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:19.900184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] PATH_INFO: `/v3/users/0e604eee245e415a8234a02438d1efc5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:19.901384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e604eee245e415a8234a02438d1efc5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:19.903377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:19.927162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:19.958225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e604eee245e415a8234a02438d1efc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.971776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e604eee245e415a8234a02438d1efc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.981849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e604eee245e415a8234a02438d1efc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.990964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e604eee245e415a8234a02438d1efc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.991696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ea8bf402-43b2-41db-a4bb-775c0c084f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e604eee245e415a8234a02438d1efc5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:19.997986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1217/2466] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:19 2022] DELETE /identity/v3/users/0e604eee245e415a8234a02438d1efc5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:20.289041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.290783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:20.306195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17613957-8c78-4829-afb2-b6d6eda8dc20 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.336209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:20.371217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.372102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.372384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.372660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] PATH_INFO: `/v3/projects/a5a1c2bc36f94f4ca140be9fbcb03d06` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.373913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a1c2bc36f94f4ca140be9fbcb03d06)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.375891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.425322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a1c2bc36f94f4ca140be9fbcb03d06', 'request_id': 'req-9769965f-a82b-40c9-8652-d5f35f5d943e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:20.431712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a1c2bc36f94f4ca140be9fbcb03d06', 'request_id': 'req-9769965f-a82b-40c9-8652-d5f35f5d943e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:20.440646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a1c2bc36f94f4ca140be9fbcb03d06', 'request_id': 'req-9769965f-a82b-40c9-8652-d5f35f5d943e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:20.451382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] The token cache is being invalidate because project a5a1c2bc36f94f4ca140be9fbcb03d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:20.451669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:20.452307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-9769965f-a82b-40c9-8652-d5f35f5d943e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:20.453873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1250/2467] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:20 2022] DELETE /identity/v3/projects/a5a1c2bc36f94f4ca140be9fbcb03d06 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:20.464778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.465065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.465343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.484934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:20.486248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:20.551492 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1218/2468] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:20.560133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.560434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.560728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.580707 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:20.582025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:20.590825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1251/2469] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:20.596529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.607631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59d68-b978-41c1-a9c2-5a2560a9163c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.615464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.616257 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.616542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.616819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.617596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:20.617904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.620667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.636525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1219/2470] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:20.641397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.652244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f43d55-0cab-4007-9210-f02d102cd753 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.658674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.659549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.659920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.660272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.661045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.663092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.694591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1252/2471] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:20 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:20.695611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.706309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c28dd-7a4d-4531-a7a3-d330f458c755 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.807562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.810664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.811108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.811400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.814508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.818912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.848626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1220/2472] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:20 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:20.853122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.867120 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be128cb-0add-4c00-b34d-4b71770381f2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.872053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.872728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.873018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.873295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.874029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.875708 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.894414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1253/2473] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:20.895614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.913717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955801-d03d-441f-a51f-9bd4c2c6e86f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.919358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95705548-56cb-4ba9-8c8b-9a0c95ad31c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.921781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95705548-56cb-4ba9-8c8b-9a0c95ad31c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.921781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95705548-56cb-4ba9-8c8b-9a0c95ad31c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.921781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95705548-56cb-4ba9-8c8b-9a0c95ad31c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.921781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95705548-56cb-4ba9-8c8b-9a0c95ad31c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.923959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95705548-56cb-4ba9-8c8b-9a0c95ad31c7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.935924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1221/2474] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:20.940194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:20.950527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e01ab8-4eca-47e1-9599-c2adcb504d81 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:20.955611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:20.956504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:20.956781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:20.957054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] PATH_INFO: `/v3/projects/9f1b72a72c624273bb1314c38f07e19a/users/4405fafd6cbb4587bf48ca33f2d892a7/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:20.973258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1b72a72c624273bb1314c38f07e19a, user_id=4405fafd6cbb4587bf48ca33f2d892a7, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:20.975955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:20.990607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1254/2475] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:20 2022] PUT /identity/v3/projects/9f1b72a72c624273bb1314c38f07e19a/users/4405fafd6cbb4587bf48ca33f2d892a7/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:21.000926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.001218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.001585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.007760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:21.032040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] MFA Rules not processed for user `4405fafd6cbb4587bf48ca33f2d892a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:21.033409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:21.123598 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1222/2476] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:21.129062 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.200160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:21.202722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2a602-33f3-48ad-b040-005223d6183b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.209401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.210621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.211150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.211544 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.212668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.215740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.248636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1255/2477] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:21 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:21.252989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.267231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e219357-1b02-4df3-80a4-e9adc21c911c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.331543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.332393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.332709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.332986 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.333693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.335727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.357810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1223/2478] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:21 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:21.361852 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.372787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1b01d-9410-4006-9164-f85af400544d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.378987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94811bd1-7efc-4395-b98c-138b9e508dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.380030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94811bd1-7efc-4395-b98c-138b9e508dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.380187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94811bd1-7efc-4395-b98c-138b9e508dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.380471 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94811bd1-7efc-4395-b98c-138b9e508dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.381500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94811bd1-7efc-4395-b98c-138b9e508dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.383420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94811bd1-7efc-4395-b98c-138b9e508dd4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.407437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1256/2479] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:21.409103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.423592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba64e-6574-4b96-bb60-e70c4734b584 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.430174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.431309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.431739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.432161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] PATH_INFO: `/v3/projects/dc2d44a1e2594373a0e1b2585639e366/users/d471095c0e4c4026ae33cba915a1c36c/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.451170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2d44a1e2594373a0e1b2585639e366, user_id=d471095c0e4c4026ae33cba915a1c36c, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.454142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.460250 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.460577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.460828 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.481244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:21.482450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:21.484247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1224/2480] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:21 2022] PUT /identity/v3/projects/dc2d44a1e2594373a0e1b2585639e366/users/d471095c0e4c4026ae33cba915a1c36c/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:21.490289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.562309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1257/2481] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:21:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:21.577807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.579909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:21.591890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:21.598051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7dbe3-7f3a-4450-8441-e7124afe3ae7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.606302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.607159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.607487 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.607764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.608655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.610300 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.625262 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1225/2482] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:21.632038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.645961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25bff-8b5d-43b4-aaa3-63e1f89e869f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.652091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c62586-9c37-4b2f-88b7-619b0175fbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.653387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62586-9c37-4b2f-88b7-619b0175fbc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.653670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62586-9c37-4b2f-88b7-619b0175fbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.653953 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62586-9c37-4b2f-88b7-619b0175fbc6 admin admin] PATH_INFO: `/v3/projects/dc2d44a1e2594373a0e1b2585639e366/users/d471095c0e4c4026ae33cba915a1c36c/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.656092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c62586-9c37-4b2f-88b7-619b0175fbc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2d44a1e2594373a0e1b2585639e366, user_id=d471095c0e4c4026ae33cba915a1c36c, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.659171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c62586-9c37-4b2f-88b7-619b0175fbc6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.671151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1226/2483] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:21 2022] PUT /identity/v3/projects/dc2d44a1e2594373a0e1b2585639e366/users/d471095c0e4c4026ae33cba915a1c36c/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:21.679782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701aa6c-9694-49b4-a20a-95c53266b7c2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.683061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.683196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.683426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.716725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] MFA Rules not processed for user `d471095c0e4c4026ae33cba915a1c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:21.717633 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:21.761451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.762690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.762896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.763159 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:21.786391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:21.816597 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1227/2484] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:21.828651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.831316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:21.847939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:21.849946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:21.865393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1258/2485] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:21:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 10:21:21.937679 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18e6b5-292e-428e-97f8-156b3e00b74a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:21.944133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:21.950323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:21.978634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:21.982585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:21.982903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:21.983187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:22.035963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:22.037501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:22.079877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1228/2486] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:22.232051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d785-e9ec-4bc8-af19-54371ea85dc1 service swift] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:22.245579 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:22.257617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:22.258448 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044ac44-fb9e-4a48-92b7-9ce100c68d47 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:22.259531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:22.261083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:22.261612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:22.266228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:22.269733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:22.270917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:22.271375 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:22.271851 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:22.416027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:22.418207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:22.448302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:22.449861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:22.468853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1259/2487] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:22.474228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1229/2488] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:22.587266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:22.590323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:22.725881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e2a7e-1f45-4b6f-af45-f3b5a9b1b1be service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:22.729999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:22.731201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:22.732031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:22.732349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:22.734528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:22.752640 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:22.755671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:22.763125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=100 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:22.773143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1260/2489] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:28.823588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:28.837163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e7223-1cf7-4a95-836f-605b8a78d755 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:28.841945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:28.842573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:28.843745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:28.844027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:28.846202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:28.857972 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:28.864654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:28.866362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb10bc-fdba-480c-9416-e39043c68b7c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:28.878873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1230/2490] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:32.410010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:32.479983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2216d42-c62e-4fb9-8900-b8de2724c24e service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:32.484962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:32.485790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:32.486071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:32.486405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] PATH_INFO: `/v3/users/4405fafd6cbb4587bf48ca33f2d892a7` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:32.487735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4405fafd6cbb4587bf48ca33f2d892a7)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:32.489712 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:32.539860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4405fafd6cbb4587bf48ca33f2d892a7'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:32.553012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4405fafd6cbb4587bf48ca33f2d892a7'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:32.560882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4405fafd6cbb4587bf48ca33f2d892a7'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:32.568476 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4405fafd6cbb4587bf48ca33f2d892a7'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:32.569202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4405fafd6cbb4587bf48ca33f2d892a7'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:32.575801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1261/2491] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:32 2022] DELETE /identity/v3/users/4405fafd6cbb4587bf48ca33f2d892a7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:21:32.581012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:32.581572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:32.581572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:32.602438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:32.604599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:32.678196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1231/2492] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:32.688260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:32.689911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:32.767305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d394c5-1b1c-429c-9b5f-616e63836752 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:32.771596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:32.772334 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:32.772619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:32.772898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:32.782622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:32.784129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:32.794743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1262/2493] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:33.391336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:33.392962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:33.408273 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6e404-dd90-4af5-938e-822952e5e5b6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:33.414469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:33.415418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:33.415687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:33.415954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] PATH_INFO: `/v3/projects/9f1b72a72c624273bb1314c38f07e19a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:33.417228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1b72a72c624273bb1314c38f07e19a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:33.419910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:33.475690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1b72a72c624273bb1314c38f07e19a', 'request_id': 'req-7d94be57-db9c-4ccd-be4c-504217b5b33f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.481776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1b72a72c624273bb1314c38f07e19a', 'request_id': 'req-7d94be57-db9c-4ccd-be4c-504217b5b33f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.488981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1b72a72c624273bb1314c38f07e19a', 'request_id': 'req-7d94be57-db9c-4ccd-be4c-504217b5b33f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.506131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] The token cache is being invalidate because project 9f1b72a72c624273bb1314c38f07e19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:33.506568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.507325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.509263 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1232/2494] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:33 2022] DELETE /identity/v3/projects/9f1b72a72c624273bb1314c38f07e19a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:33.513059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:33.525467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ea333-b313-48be-9110-38b2270c33df service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:33.546839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:33.593111 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:33.594633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:33.595551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:33.595935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] PATH_INFO: `/v3/users/d471095c0e4c4026ae33cba915a1c36c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:33.597915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d471095c0e4c4026ae33cba915a1c36c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:33.600019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:33.660576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd471095c0e4c4026ae33cba915a1c36c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.678635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd471095c0e4c4026ae33cba915a1c36c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.687876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd471095c0e4c4026ae33cba915a1c36c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.696998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd471095c0e4c4026ae33cba915a1c36c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.698018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-2af86bc7-c96a-4167-9c1f-e3c043838469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd471095c0e4c4026ae33cba915a1c36c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:33.708029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1263/2495] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:33 2022] DELETE /identity/v3/users/d471095c0e4c4026ae33cba915a1c36c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:34.014655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.016770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:34.040595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94be57-db9c-4ccd-be4c-504217b5b33f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.120140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.121132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.121466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.121809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] PATH_INFO: `/v3/projects/dc2d44a1e2594373a0e1b2585639e366` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.123092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2d44a1e2594373a0e1b2585639e366)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.125553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.148585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:34.182036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2d44a1e2594373a0e1b2585639e366', 'request_id': 'req-33c9e802-2317-42c9-8814-55a67c77bb29'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:34.189198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2d44a1e2594373a0e1b2585639e366', 'request_id': 'req-33c9e802-2317-42c9-8814-55a67c77bb29'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:34.195880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2d44a1e2594373a0e1b2585639e366', 'request_id': 'req-33c9e802-2317-42c9-8814-55a67c77bb29'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:34.210872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] The token cache is being invalidate because project dc2d44a1e2594373a0e1b2585639e366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:34.211176 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:34.211808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-33c9e802-2317-42c9-8814-55a67c77bb29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:34.214179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1233/2496] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:34 2022] DELETE /identity/v3/projects/dc2d44a1e2594373a0e1b2585639e366 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:34.219920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48606991-4852-4832-93bb-a807049eef2b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.220196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48606991-4852-4832-93bb-a807049eef2b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.220499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48606991-4852-4832-93bb-a807049eef2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.239794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-48606991-4852-4832-93bb-a807049eef2b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:34.241033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-48606991-4852-4832-93bb-a807049eef2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:34.302034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1264/2497] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:34.309962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.310255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.310531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.330396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:34.331723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:34.342583 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1234/2498] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:34.347223 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48606991-4852-4832-93bb-a807049eef2b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.358323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48606991-4852-4832-93bb-a807049eef2b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.364910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.365695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.366049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.366397 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.367248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:34.367627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.369501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.386744 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1265/2499] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:34.389593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.399815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0103f-50b4-4cf8-b960-37a1bd7ee6a7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.406146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.406857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.407139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.407519 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.408148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.410311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.444004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1235/2500] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:34 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:34.444850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.460577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71816f2-f475-46e4-b85c-d5a69fd73fcd admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.506523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:34.555157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.555835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.556303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.556580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.557286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.559274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.584542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1266/2501] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:34 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:34.588712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.601270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83b7fb-a205-4543-987b-748c405bf0a2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.616943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.617644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.617837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.618105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.619362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.620498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.639452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1236/2502] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:34.640082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.651395 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7c6e-c9de-4c19-9cef-a57f5890c890 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.659148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0377b00-1f7b-4c3d-bf07-00057d0f0c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.660152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0377b00-1f7b-4c3d-bf07-00057d0f0c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.660623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0377b00-1f7b-4c3d-bf07-00057d0f0c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.661053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0377b00-1f7b-4c3d-bf07-00057d0f0c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.661985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0377b00-1f7b-4c3d-bf07-00057d0f0c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.663652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0377b00-1f7b-4c3d-bf07-00057d0f0c7e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.676304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1267/2503] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:34.680621 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.691329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480f90-ca98-4aa8-8c51-ddb183506c87 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.699673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.700702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.701042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.701377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] PATH_INFO: `/v3/projects/8a3d2f54acaa45f297d9127485924506/users/01237f83fa0943d0ac85d235b53920a0/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.725163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3d2f54acaa45f297d9127485924506, user_id=01237f83fa0943d0ac85d235b53920a0, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.729452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:34.747158 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1237/2504] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:34 2022] PUT /identity/v3/projects/8a3d2f54acaa45f297d9127485924506/users/01237f83fa0943d0ac85d235b53920a0/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:34.758036 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.759219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.759219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.787914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] MFA Rules not processed for user `01237f83fa0943d0ac85d235b53920a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:34.789142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:34.865719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1268/2505] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:34.870607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:34.940433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d95c2-7df0-445d-8629-7aa010e60daa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:34.946533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:34.947385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:34.947648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:34.947916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:34.948631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:34.950556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.121936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1238/2506] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:34 2022] POST /identity/v3/projects => generated 367 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:35.126392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:35.145056 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37630fa3-f65e-4383-896e-ff491b283541 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:35.218183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:35.218793 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.219078 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.219352 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.220046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:35.222014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.245034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1269/2507] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:35 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:35.250237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:35.261416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400063bf-899c-42ca-8d09-c41db130c8b6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:35.266634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:35.267343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.267625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.267900 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.268640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:35.270370 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.282536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1239/2508] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:35.287166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:35.297422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808812-1c2a-4885-8ed6-99f2add4142e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:35.302128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:35.303046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.303344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.303616 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] PATH_INFO: `/v3/projects/f417fc985bef49b9ad995f2bf5b07457/users/652ac3564ba24394814e08503d5f256e/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.318790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f417fc985bef49b9ad995f2bf5b07457, user_id=652ac3564ba24394814e08503d5f256e, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:35.321465 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.331907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:35.337715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1270/2509] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:35 2022] PUT /identity/v3/projects/f417fc985bef49b9ad995f2bf5b07457/users/652ac3564ba24394814e08503d5f256e/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:35.342152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:35.352278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2ab8-3025-4b36-b575-cab6b3e4899e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:35.364236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:35.409971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ce052-0e48-4012-a00b-0befc714a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:35.410632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce052-0e48-4012-a00b-0befc714a854 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.410941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce052-0e48-4012-a00b-0befc714a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.411213 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce052-0e48-4012-a00b-0befc714a854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.411939 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ce052-0e48-4012-a00b-0befc714a854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:35.413555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ce052-0e48-4012-a00b-0befc714a854 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.424163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1240/2510] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:35.430951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:35.444373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478415c6-d24c-405f-b214-e91cec09c47b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:35.450635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:35.451635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.451922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.452197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] PATH_INFO: `/v3/projects/f417fc985bef49b9ad995f2bf5b07457/users/652ac3564ba24394814e08503d5f256e/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.454399 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f417fc985bef49b9ad995f2bf5b07457, user_id=652ac3564ba24394814e08503d5f256e, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:35.457297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.470268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1271/2511] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:35 2022] PUT /identity/v3/projects/f417fc985bef49b9ad995f2bf5b07457/users/652ac3564ba24394814e08503d5f256e/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:35.483165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.483456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.483735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.511113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] MFA Rules not processed for user `652ac3564ba24394814e08503d5f256e`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:35.512379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:35.585139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1241/2512] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:35.597342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:35.599186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:35.689042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93e1ca-c2c3-4cbf-ac6d-a73ccaca0476 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:35.703638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eb2fa-538d-4471-a90a-3657d6cb8a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:35.711550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb2fa-538d-4471-a90a-3657d6cb8a58 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:35.712025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb2fa-538d-4471-a90a-3657d6cb8a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:35.712325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb2fa-538d-4471-a90a-3657d6cb8a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:35.726529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eb2fa-538d-4471-a90a-3657d6cb8a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:35.727950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eb2fa-538d-4471-a90a-3657d6cb8a58 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:35.739537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1272/2513] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:37.161807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:37.225247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7e388-5429-4810-9691-6a13790350a3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:37.230838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:37.231807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:37.232343 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:37.232620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] PATH_INFO: `/v3/users/01237f83fa0943d0ac85d235b53920a0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:37.233825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=01237f83fa0943d0ac85d235b53920a0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:37.236061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:37.281141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:37.287523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01237f83fa0943d0ac85d235b53920a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.300795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01237f83fa0943d0ac85d235b53920a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.309021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01237f83fa0943d0ac85d235b53920a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.315967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01237f83fa0943d0ac85d235b53920a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.316645 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01237f83fa0943d0ac85d235b53920a0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.322613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1242/2514] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:37 2022] DELETE /identity/v3/users/01237f83fa0943d0ac85d235b53920a0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:37.327249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:37.327779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:37.327890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:37.348103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:37.349329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:37.416481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1273/2515] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:37.426408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:37.428447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:37.493124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d47a3-2fa3-40f4-acb0-3a9b7c1381fd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:37.497410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:37.498059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:37.498341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:37.498620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:37.508604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:37.510062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:37.523088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1243/2516] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:37.805568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:37.807109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:37.820437 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35397b-c394-4aa1-afd5-a6d4a685f0ae None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:37.825088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:37.826057 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:37.826403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:37.826765 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] PATH_INFO: `/v3/projects/8a3d2f54acaa45f297d9127485924506` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:37.828117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3d2f54acaa45f297d9127485924506)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:37.830181 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:37.878633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3d2f54acaa45f297d9127485924506', 'request_id': 'req-5fb5c330-e508-4e9a-9135-4e2841a262ba'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.884835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3d2f54acaa45f297d9127485924506', 'request_id': 'req-5fb5c330-e508-4e9a-9135-4e2841a262ba'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.894215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3d2f54acaa45f297d9127485924506', 'request_id': 'req-5fb5c330-e508-4e9a-9135-4e2841a262ba'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.905488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] The token cache is being invalidate because project 8a3d2f54acaa45f297d9127485924506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:37.905777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.906452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:37.908053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1274/2517] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:37 2022] DELETE /identity/v3/projects/8a3d2f54acaa45f297d9127485924506 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:37.913848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:37.928826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a2dec-f395-4179-8cd4-4f1765a2f414 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:37.990433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:37.991345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:37.991630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:37.991890 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] PATH_INFO: `/v3/users/652ac3564ba24394814e08503d5f256e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:37.993083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] RBAC: Authorizing `identity:delete_user(user_id=652ac3564ba24394814e08503d5f256e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:37.995114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:38.046961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '652ac3564ba24394814e08503d5f256e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.060042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:38.061773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '652ac3564ba24394814e08503d5f256e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.072441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '652ac3564ba24394814e08503d5f256e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.080719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '652ac3564ba24394814e08503d5f256e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.081489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f8785c69-d5e9-4b9d-91bb-c0fc435faeda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '652ac3564ba24394814e08503d5f256e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.088576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1244/2518] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:37 2022] DELETE /identity/v3/users/652ac3564ba24394814e08503d5f256e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:38.445444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:38.446959 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:38.460328 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5c330-e508-4e9a-9135-4e2841a262ba admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:38.470150 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:38.529878 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:38.531082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.531368 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.531645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] PATH_INFO: `/v3/projects/f417fc985bef49b9ad995f2bf5b07457` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.532861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f417fc985bef49b9ad995f2bf5b07457)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:38.534824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:38.582283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f417fc985bef49b9ad995f2bf5b07457', 'request_id': 'req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.588738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f417fc985bef49b9ad995f2bf5b07457', 'request_id': 'req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.596681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f417fc985bef49b9ad995f2bf5b07457', 'request_id': 'req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.608965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] The token cache is being invalidate because project f417fc985bef49b9ad995f2bf5b07457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:38.609247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.609881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f7256164-4f5c-48e1-b9f2-a9b484ec6e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:38.611830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1275/2519] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:38 2022] DELETE /identity/v3/projects/f417fc985bef49b9ad995f2bf5b07457 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:38.616622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.617146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.617264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.637848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:38.639107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:38.703802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1245/2520] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:38.711655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.711943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.712228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.733047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:38.734273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:38.744336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1276/2521] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:38.749112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:38.759599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8b2b-da44-426a-82c3-35b7f8c7a5c5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:38.764507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:38.765143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.765572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.765850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.766594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:38.766906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:38.768542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:38.780414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1246/2522] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:38.784867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:38.794998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759002c-c093-42eb-b74d-5aedb6a94b6f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:38.800897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:38.801641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.801991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.802363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.803172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:38.805162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:38.846659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1277/2523] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:38 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:38.851048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:38.860966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e586d-4a87-411a-a01f-852547a2184e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:38.919038 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:38.919723 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.920007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.920285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.920984 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:38.922963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:38.952811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1247/2524] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:38 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:38.957638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:38.970734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf1801-7223-4ac5-b546-48e0c42f5b78 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:38.978347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:38.979112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:38.979391 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:38.979665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:38.980508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:38.982254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:38.994151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1278/2525] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:38.998821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.010994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e150-8318-43bf-86a3-1f96f78939c4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.016052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9628e-d8c1-4ec6-98c0-8f87e92d577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.016720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9628e-d8c1-4ec6-98c0-8f87e92d577f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.017046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9628e-d8c1-4ec6-98c0-8f87e92d577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.017329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9628e-d8c1-4ec6-98c0-8f87e92d577f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.018190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9628e-d8c1-4ec6-98c0-8f87e92d577f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.019856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9628e-d8c1-4ec6-98c0-8f87e92d577f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.030963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1248/2526] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:39.035349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.046409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110df5e6-0751-4dc2-b86f-5f1684826f34 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.052284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.053210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.053574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.053804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] PATH_INFO: `/v3/projects/f6402deb5a3c4dcf8022f36442459969/users/7be150c429304767b7fec1b94cf745e5/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.070722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6402deb5a3c4dcf8022f36442459969, user_id=7be150c429304767b7fec1b94cf745e5, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.074242 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.095444 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1279/2527] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:39 2022] PUT /identity/v3/projects/f6402deb5a3c4dcf8022f36442459969/users/7be150c429304767b7fec1b94cf745e5/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:39.106911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.107345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.107639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.143034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] MFA Rules not processed for user `7be150c429304767b7fec1b94cf745e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:39.144295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:39.183034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:39.217604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1249/2528] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:39.224438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.274000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:39.305106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1026d0-4071-4355-afee-4e66977d51af admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.310439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.311134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.311435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.311706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.312400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.314446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.343748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1280/2529] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:39 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:39.348475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.363452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51761a75-a0fe-43ce-be3c-0c9d77f0bd13 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.425382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.426725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.427039 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.427313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.428132 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.430168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.450831 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1250/2530] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:39 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:39.454979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.466327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7388-e4c6-48ed-8afd-dddb31a001b4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.471963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.472625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.473051 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.473289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.474214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.475866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.487059 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1281/2531] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:39.491617 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.502988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171381d1-ee6d-4fe0-9ed1-d194cd200c0e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.507979 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.508857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.509139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.509410 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] PATH_INFO: `/v3/projects/acb7d0d7e05348cabe068d9dfdbd119d/users/4d5b0bd473a44d928b8b6cf27b706bbf/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.525407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb7d0d7e05348cabe068d9dfdbd119d, user_id=4d5b0bd473a44d928b8b6cf27b706bbf, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.528174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.543624 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1251/2532] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:39 2022] PUT /identity/v3/projects/acb7d0d7e05348cabe068d9dfdbd119d/users/4d5b0bd473a44d928b8b6cf27b706bbf/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:39.547967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.558568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c2476-c8fc-450d-aa43-02cc3e3c1b20 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.639268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e58cec-2c67-47a2-a9f7-a88ebb352f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.639942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e58cec-2c67-47a2-a9f7-a88ebb352f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.640229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e58cec-2c67-47a2-a9f7-a88ebb352f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.640509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e58cec-2c67-47a2-a9f7-a88ebb352f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.641275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e58cec-2c67-47a2-a9f7-a88ebb352f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.642934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e58cec-2c67-47a2-a9f7-a88ebb352f33 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.653447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1282/2533] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:39.657775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.668766 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255525-d140-4a34-b93a-cc9dfd8674df admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.673944 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.674853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.675137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.675409 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] PATH_INFO: `/v3/projects/acb7d0d7e05348cabe068d9dfdbd119d/users/4d5b0bd473a44d928b8b6cf27b706bbf/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.677506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb7d0d7e05348cabe068d9dfdbd119d, user_id=4d5b0bd473a44d928b8b6cf27b706bbf, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.680259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.691764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1252/2534] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:39 2022] PUT /identity/v3/projects/acb7d0d7e05348cabe068d9dfdbd119d/users/4d5b0bd473a44d928b8b6cf27b706bbf/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:39.702224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.702613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.702918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.730951 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] MFA Rules not processed for user `4d5b0bd473a44d928b8b6cf27b706bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:39.732177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:39.805244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1283/2535] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:39.818198 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:39.820200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:39.894548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88797864-f6d9-444c-8c92-e3335dea7c24 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:39.898493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:39.899179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:39.899462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:39.899735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:39.909082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:39.910940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:39.920503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1253/2536] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:40.274863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:40.287587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d11f65-2adc-4cef-a2bd-9cef33ddd68e None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:40.293215 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d5b0bd473a44d928b8b6cf27b706bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acb7d0d7e05348cabe068d9dfdbd119d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:40.293892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:40.294183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:40.294463 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:40.295240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:40.295555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:40.297244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:40.310788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1284/2537] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:40.314875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:40.324653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdb605-2959-4669-b466-8d753d325a9b service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:40.330123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d5b0bd473a44d928b8b6cf27b706bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acb7d0d7e05348cabe068d9dfdbd119d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:40.330849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:40.331137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:40.331421 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:40.332151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:40.334152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:40.364660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1254/2538] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:40 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:40.365941 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:40.377558 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff50656-8444-4497-99ae-80a9106798d4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:40.398604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:40.440346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d5b0bd473a44d928b8b6cf27b706bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acb7d0d7e05348cabe068d9dfdbd119d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:40.441066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:40.441540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:40.441918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:40.443034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:40.443859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:40.445483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:40.457284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1285/2539] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:40.495627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:40.506750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20542b05-95cd-4609-bf37-2dbb62a2bf86 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:40.512094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d5b0bd473a44d928b8b6cf27b706bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acb7d0d7e05348cabe068d9dfdbd119d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:40.513046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:40.513358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:40.514019 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] PATH_INFO: `/v3/projects/eb28a17760d04ba987ac63385806e59d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:40.521005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eb28a17760d04ba987ac63385806e59d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:40.523027 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:40.525704 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:40.582053 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb28a17760d04ba987ac63385806e59d', 'request_id': 'req-bf8a18b7-5622-441b-bb51-ec74c58a49cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:40.587640 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb28a17760d04ba987ac63385806e59d', 'request_id': 'req-bf8a18b7-5622-441b-bb51-ec74c58a49cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:40.594650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb28a17760d04ba987ac63385806e59d', 'request_id': 'req-bf8a18b7-5622-441b-bb51-ec74c58a49cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:40.608751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] The token cache is being invalidate because project eb28a17760d04ba987ac63385806e59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:40.609074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:40.609832 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf8a18b7-5622-441b-bb51-ec74c58a49cc tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:40.611639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1255/2540] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:40 2022] DELETE /identity/v3/projects/eb28a17760d04ba987ac63385806e59d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:40.830148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:40.832103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:40.900550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040324b-8264-4ba1-a15d-ab491a1e4ee4 tempest-VolumeQuotaClassesTest-465926744 tempest-VolumeQuotaClassesTest-465926744-project-admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:40.967638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:40.968440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:40.968715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:40.968993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] PATH_INFO: `/v3/users/7be150c429304767b7fec1b94cf745e5` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:40.970266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be150c429304767b7fec1b94cf745e5)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:40.972498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:41.024543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be150c429304767b7fec1b94cf745e5'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.036672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be150c429304767b7fec1b94cf745e5'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.038370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:41.045192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be150c429304767b7fec1b94cf745e5'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.053738 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be150c429304767b7fec1b94cf745e5'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.054518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be150c429304767b7fec1b94cf745e5'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.061939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1286/2541] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:40 2022] DELETE /identity/v3/users/7be150c429304767b7fec1b94cf745e5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:41.066806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:41.067315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:41.067411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:41.086155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:41.087408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:41.152962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1256/2542] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:41.163535 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:41.165870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:41.234441 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182d410-a00d-4ddb-b63f-47f7fd5e4cc7 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:41.238402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:41.239369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:41.239663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:41.239942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:41.249885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:41.251378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:41.261833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1287/2543] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:41.564481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:41.565972 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:41.581557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2415e-c75d-47d5-8bdc-837687a650dd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:41.586626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:41.587550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:41.587830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:41.588454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] PATH_INFO: `/v3/projects/f6402deb5a3c4dcf8022f36442459969` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:41.589677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6402deb5a3c4dcf8022f36442459969)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:41.591709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:41.645726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6402deb5a3c4dcf8022f36442459969', 'request_id': 'req-fae9ba4d-c46b-4483-896e-c45966330962'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.652111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6402deb5a3c4dcf8022f36442459969', 'request_id': 'req-fae9ba4d-c46b-4483-896e-c45966330962'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.658591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6402deb5a3c4dcf8022f36442459969', 'request_id': 'req-fae9ba4d-c46b-4483-896e-c45966330962'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.679026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] The token cache is being invalidate because project f6402deb5a3c4dcf8022f36442459969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:41.679444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.680207 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.681958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1257/2544] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:41 2022] DELETE /identity/v3/projects/f6402deb5a3c4dcf8022f36442459969 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:41.686224 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:41.699533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bc978-b8f4-4d00-a27d-dd89a7416ca0 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:41.758873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:41.759764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:41.760046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:41.760319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] PATH_INFO: `/v3/users/4d5b0bd473a44d928b8b6cf27b706bbf` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:41.761536 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5b0bd473a44d928b8b6cf27b706bbf)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:41.763505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:41.809620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5b0bd473a44d928b8b6cf27b706bbf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.822270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5b0bd473a44d928b8b6cf27b706bbf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.831587 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5b0bd473a44d928b8b6cf27b706bbf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.839146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5b0bd473a44d928b8b6cf27b706bbf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.839811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f4755adc-4dc7-4841-82ac-6c7885c8f4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5b0bd473a44d928b8b6cf27b706bbf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:41.845912 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1288/2545] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:41 2022] DELETE /identity/v3/users/4d5b0bd473a44d928b8b6cf27b706bbf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:42.146901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.148534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:42.162061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9ba4d-c46b-4483-896e-c45966330962 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.180755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:42.229698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.230531 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.230835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.231102 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] PATH_INFO: `/v3/projects/acb7d0d7e05348cabe068d9dfdbd119d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.232318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb7d0d7e05348cabe068d9dfdbd119d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.234250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.286791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb7d0d7e05348cabe068d9dfdbd119d', 'request_id': 'req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:42.292142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb7d0d7e05348cabe068d9dfdbd119d', 'request_id': 'req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:42.298192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb7d0d7e05348cabe068d9dfdbd119d', 'request_id': 'req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:42.312165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] The token cache is being invalidate because project acb7d0d7e05348cabe068d9dfdbd119d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:42.312447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:42.313066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22e91d40-d5b5-41ab-85f1-a3a5db62ea60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:42.314614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1258/2546] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:42 2022] DELETE /identity/v3/projects/acb7d0d7e05348cabe068d9dfdbd119d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:42.320322 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.320610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.320887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.328641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:42.342788 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:42.344645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:42.404808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1289/2547] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:42.413737 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.414117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.414471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.436907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:42.438118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:42.446266 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1259/2548] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:42.450563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.460594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7ad97-5d0a-4a55-a63b-d0d1a6ad168f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.465486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.466125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.466410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.466683 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.467447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:42.467753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.469362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.481370 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1290/2549] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:42.485557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.495764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b65c3-ee35-4809-81b9-ed9d386aca35 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.501250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.501884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.502160 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.502443 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.503187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.505162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.531397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1260/2550] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:42.535666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.546779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862e510-855c-4d8a-913f-8b7808c694a8 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.608604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.609315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.609629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.609905 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.610607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.612606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.633694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1291/2551] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:42.637913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.648442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cd9f2-7483-476d-b558-6634e3822d72 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.653386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.654079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.654360 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.654696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.655486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.657110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.668380 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1261/2552] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:42.672411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.683265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c88d2-08dc-4a5c-99a1-f6b0bb554db2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.688203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b579d-a0bd-48a3-ae26-f398ad8fff38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.688884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b579d-a0bd-48a3-ae26-f398ad8fff38 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.689186 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b579d-a0bd-48a3-ae26-f398ad8fff38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.689451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b579d-a0bd-48a3-ae26-f398ad8fff38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.690190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b579d-a0bd-48a3-ae26-f398ad8fff38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.692554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b579d-a0bd-48a3-ae26-f398ad8fff38 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.703638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1292/2553] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:42.708112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.719196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a1efd-7664-47e3-bd60-21225651eda9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.724533 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.725434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.725717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.725990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] PATH_INFO: `/v3/projects/e4abce9d55b14c3184840831a6e112b8/users/94b438e6a78443378ae9f0c5bfad78b8/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.741756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4abce9d55b14c3184840831a6e112b8, user_id=94b438e6a78443378ae9f0c5bfad78b8, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.744468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.760507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1262/2554] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:42 2022] PUT /identity/v3/projects/e4abce9d55b14c3184840831a6e112b8/users/94b438e6a78443378ae9f0c5bfad78b8/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:42.771835 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.772122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.772401 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.800741 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] MFA Rules not processed for user `94b438e6a78443378ae9f0c5bfad78b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:42.801321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:42.866724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:42.871897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1293/2555] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:42.876369 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:42.922738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:42.942801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07796b0-bbf0-4d6f-bc0f-c159b3e15482 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:42.947932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:42.948575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:42.948867 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:42.949137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:42.949827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:42.951782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:42.985678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1263/2556] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:42.990321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.001232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00a546-f86e-4e9d-b899-72af55a62745 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.057206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.057804 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.058086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.058360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.059103 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.061047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.086074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1294/2557] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:42 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:43.091956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.103532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3214b1-fc4b-44cc-a8ea-633d086995d6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.108459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.109110 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.109391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.109665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.110397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.112068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.122845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1264/2558] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:43.127333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.138734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f074-efcd-471f-a517-d3ede280e8d1 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.143936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.144920 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.145170 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.145439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] PATH_INFO: `/v3/projects/d47b8a93f7734a75ab8a9a05dc8be660/users/7e4f4626f27c4f80a4a28dcdda0b50b8/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.160445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47b8a93f7734a75ab8a9a05dc8be660, user_id=7e4f4626f27c4f80a4a28dcdda0b50b8, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.163140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.178829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1295/2559] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:43 2022] PUT /identity/v3/projects/d47b8a93f7734a75ab8a9a05dc8be660/users/7e4f4626f27c4f80a4a28dcdda0b50b8/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:43.183275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.193719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467f65-a291-49aa-918e-78156f5ba94e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.255275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47af8c-f52d-4f0c-87c5-6653b347ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.255958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47af8c-f52d-4f0c-87c5-6653b347ab12 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.256239 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47af8c-f52d-4f0c-87c5-6653b347ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.256501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47af8c-f52d-4f0c-87c5-6653b347ab12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.257260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47af8c-f52d-4f0c-87c5-6653b347ab12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.258963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47af8c-f52d-4f0c-87c5-6653b347ab12 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.270170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1265/2560] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:43.274914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.285929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6412-2294-4947-bba3-7f4e8cd74d87 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.291178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.292091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.292376 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.292652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] PATH_INFO: `/v3/projects/d47b8a93f7734a75ab8a9a05dc8be660/users/7e4f4626f27c4f80a4a28dcdda0b50b8/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.294807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47b8a93f7734a75ab8a9a05dc8be660, user_id=7e4f4626f27c4f80a4a28dcdda0b50b8, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.297560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.308739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1296/2561] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:43 2022] PUT /identity/v3/projects/d47b8a93f7734a75ab8a9a05dc8be660/users/7e4f4626f27c4f80a4a28dcdda0b50b8/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:43.319212 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c87649-915c-491c-9b3a-1675395de623 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.319499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c87649-915c-491c-9b3a-1675395de623 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.319777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c87649-915c-491c-9b3a-1675395de623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.348746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-14c87649-915c-491c-9b3a-1675395de623 None None] MFA Rules not processed for user `7e4f4626f27c4f80a4a28dcdda0b50b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:43.349956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-14c87649-915c-491c-9b3a-1675395de623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:43.432968 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1266/2562] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:43.445526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.447365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:43.508673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89153e3b-670f-4b9c-963a-945aaddfe8a4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.513138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.513812 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.514087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.514377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.525786 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.527272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.536754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1297/2563] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:43.584097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c87649-915c-491c-9b3a-1675395de623 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:43.585819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-14c87649-915c-491c-9b3a-1675395de623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:43.599698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c87649-915c-491c-9b3a-1675395de623 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:43.603643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9eea0-4e4b-4212-b6e3-20c1203c3113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:43.604287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9eea0-4e4b-4212-b6e3-20c1203c3113 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:43.604568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9eea0-4e4b-4212-b6e3-20c1203c3113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:43.604838 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9eea0-4e4b-4212-b6e3-20c1203c3113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:43.640217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:43.671908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9eea0-4e4b-4212-b6e3-20c1203c3113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:43.673393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9eea0-4e4b-4212-b6e3-20c1203c3113 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:43.682899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1267/2564] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:47.064688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:47.126266 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:47.133607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4a341-5b39-47d5-add5-9ac8ec5e315f service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:47.138608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:47.139526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:47.139810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:47.140088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] PATH_INFO: `/v3/users/94b438e6a78443378ae9f0c5bfad78b8` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:47.141302 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b438e6a78443378ae9f0c5bfad78b8)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:47.143417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:47.193420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b438e6a78443378ae9f0c5bfad78b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.205803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b438e6a78443378ae9f0c5bfad78b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.214542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b438e6a78443378ae9f0c5bfad78b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.222130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b438e6a78443378ae9f0c5bfad78b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.222837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b438e6a78443378ae9f0c5bfad78b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.229526 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1298/2565] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:47 2022] DELETE /identity/v3/users/94b438e6a78443378ae9f0c5bfad78b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:47.234383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:47.234928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:47.234928 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:47.253260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:47.254467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:47.319915 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1268/2566] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:47.329878 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:47.331462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:47.391178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f3f6-bde8-478a-b4ef-1d8a6a13188e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:47.395628 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:47.396289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:47.396573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:47.396851 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:47.406750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:47.408199 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:47.418176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1299/2567] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:47.695523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:47.697113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:47.712785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364854-3237-4efc-8f96-c6d1b465052b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:47.717913 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:47.719088 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:47.719384 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:47.719654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] PATH_INFO: `/v3/projects/e4abce9d55b14c3184840831a6e112b8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:47.721012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4abce9d55b14c3184840831a6e112b8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:47.723064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:47.779666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4abce9d55b14c3184840831a6e112b8', 'request_id': 'req-bee27236-12e9-4fc7-b11e-e39c22613c13'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.785676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4abce9d55b14c3184840831a6e112b8', 'request_id': 'req-bee27236-12e9-4fc7-b11e-e39c22613c13'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.792315 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4abce9d55b14c3184840831a6e112b8', 'request_id': 'req-bee27236-12e9-4fc7-b11e-e39c22613c13'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.806604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] The token cache is being invalidate because project e4abce9d55b14c3184840831a6e112b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:47.806995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.807786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.809727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1269/2568] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:47 2022] DELETE /identity/v3/projects/e4abce9d55b14c3184840831a6e112b8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:47.813953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:47.824503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceddf2e-03c8-4770-8474-ec69cca5ca3f service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:47.886304 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:47.887211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:47.887499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:47.887777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] PATH_INFO: `/v3/users/7e4f4626f27c4f80a4a28dcdda0b50b8` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:47.889018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4f4626f27c4f80a4a28dcdda0b50b8)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:47.891014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:47.912958 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:47.943077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4f4626f27c4f80a4a28dcdda0b50b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.954909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4f4626f27c4f80a4a28dcdda0b50b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.964427 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4f4626f27c4f80a4a28dcdda0b50b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.972323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4f4626f27c4f80a4a28dcdda0b50b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.973012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4f4626f27c4f80a4a28dcdda0b50b8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:47.979556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1300/2569] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:47 2022] DELETE /identity/v3/users/7e4f4626f27c4f80a4a28dcdda0b50b8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:48.270072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.271841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.285141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee27236-12e9-4fc7-b11e-e39c22613c13 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.348891 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.349781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.350062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.350334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] PATH_INFO: `/v3/projects/d47b8a93f7734a75ab8a9a05dc8be660` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.351657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47b8a93f7734a75ab8a9a05dc8be660)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.353585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.355275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:48.403614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.405222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.412185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47b8a93f7734a75ab8a9a05dc8be660', 'request_id': 'req-b12360b3-2217-4ea5-adb3-d944533bf6a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:48.417956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47b8a93f7734a75ab8a9a05dc8be660', 'request_id': 'req-b12360b3-2217-4ea5-adb3-d944533bf6a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:48.424975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47b8a93f7734a75ab8a9a05dc8be660', 'request_id': 'req-b12360b3-2217-4ea5-adb3-d944533bf6a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:48.440058 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] The token cache is being invalidate because project d47b8a93f7734a75ab8a9a05dc8be660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:48.440344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:48.440970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b12360b3-2217-4ea5-adb3-d944533bf6a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:48.442861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1270/2570] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:48 2022] DELETE /identity/v3/projects/d47b8a93f7734a75ab8a9a05dc8be660 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:48.447507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea776ff7-b86b-4937-8ad7-a915f1548558 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.447800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea776ff7-b86b-4937-8ad7-a915f1548558 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.448078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea776ff7-b86b-4937-8ad7-a915f1548558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.465206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9841f9-efb4-47c8-9c95-d71092ab1fe2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.467344 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ea776ff7-b86b-4937-8ad7-a915f1548558 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:48.468525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ea776ff7-b86b-4937-8ad7-a915f1548558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.514983 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.515695 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.515979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.516252 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.518363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.528029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1271/2571] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:48.537228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.537511 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.537782 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.557049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:48.558217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.566377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1272/2572] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:21:48.571169 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.580118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed711-8f59-4dff-9abf-3515554299e4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.583864 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.584822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.585355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.585488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.585750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.586050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.586806 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:48.587108 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.589098 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.595860 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1301/2573] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:48.601713 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1273/2574] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:21:48.605903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.611115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:48.617790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323714d-7b30-443a-9ba2-d5a0d94e03e2 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.623785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.624461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.624746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.625028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.625743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.627737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.657577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1302/2575] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:48 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:48.662187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.672433 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3e85-3de2-47a2-afec-4bdce688ac06 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.745206 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.745895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.746181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.746515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.747357 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.749391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.768170 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:48.777399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1274/2576] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:48 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:48.778625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.790140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ec84-ccdb-4bc8-81ec-f07ab7f048e1 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.795346 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.796141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.796408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.796686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.797508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.799238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.812347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1303/2577] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:48.816422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.828726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf59c19-3e01-4907-aab4-a9c8e8bdc46e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.834652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.844561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.845036 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.845411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.846481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.848113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.862494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1275/2578] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:48.863622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.877742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6b819-a367-42ed-9305-25758ebd1f8f admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:48.883771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c01ae-c630-44c1-b3be-0f25e282ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:48.884767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c01ae-c630-44c1-b3be-0f25e282ac15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.885104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c01ae-c630-44c1-b3be-0f25e282ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.885446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c01ae-c630-44c1-b3be-0f25e282ac15 admin admin] PATH_INFO: `/v3/projects/4da70866d1504984ba191ea8c449e0bb/users/c0ec565411a5432b930433286ce9c09c/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.899021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:48.901096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.903068 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c01ae-c630-44c1-b3be-0f25e282ac15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da70866d1504984ba191ea8c449e0bb, user_id=c0ec565411a5432b930433286ce9c09c, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:48.905792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c01ae-c630-44c1-b3be-0f25e282ac15 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:48.929691 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1304/2579] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:48 2022] PUT /identity/v3/projects/4da70866d1504984ba191ea8c449e0bb/users/c0ec565411a5432b930433286ce9c09c/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:48.932227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:48.932919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:48.933210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:48.962509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] MFA Rules not processed for user `c0ec565411a5432b930433286ce9c09c`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:48.964390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:48.984868 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552403f-7718-417c-a505-e2868cb038ee admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.056531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1305/2580] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:49.057234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.064508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.065501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.066208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.066208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.068152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:49.133528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a09a-b269-4c6d-b8c3-bfe4f8237632 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.138660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.139359 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.139645 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.139928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.140888 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.144029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.144180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.145841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.155955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1276/2581] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:49.181939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1306/2582] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:49 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:49.183202 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.195403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2151365-8365-417f-be32-c7144c1b8852 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.220884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:49.263912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.264711 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.265005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.265277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.266117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.268134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.289982 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1277/2583] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:49 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:49.294560 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.307571 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a40c1-75e4-4b6b-9238-253a434eef86 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.312837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.313550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.314099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.314099 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.314927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.316472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.330856 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1307/2584] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:49.336107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.348364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ddc37-f9ce-4d2a-831f-d83e1d0f875f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.353226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.354116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.354401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.354672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] PATH_INFO: `/v3/projects/bf9596c1f26040489f57d9de1ea8f31a/users/c19f0e6e4d9240fbaeaf155e81b59853/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.372922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9596c1f26040489f57d9de1ea8f31a, user_id=c19f0e6e4d9240fbaeaf155e81b59853, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.375775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.391407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1278/2585] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:49 2022] PUT /identity/v3/projects/bf9596c1f26040489f57d9de1ea8f31a/users/c19f0e6e4d9240fbaeaf155e81b59853/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:49.395585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.406239 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d94b5-b665-4286-8948-8cd4102c1ae5 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.412107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:49.470110 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76f5a9-0aa2-4662-8a08-4ac4807c81c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.470823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f5a9-0aa2-4662-8a08-4ac4807c81c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.471105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f5a9-0aa2-4662-8a08-4ac4807c81c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.471379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f5a9-0aa2-4662-8a08-4ac4807c81c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.472105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76f5a9-0aa2-4662-8a08-4ac4807c81c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.474469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76f5a9-0aa2-4662-8a08-4ac4807c81c4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.484863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1308/2586] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:49.490741 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.502367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2776f-de20-4cd9-a972-56e1731ce316 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.513078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.514196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.514626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.514964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] PATH_INFO: `/v3/projects/bf9596c1f26040489f57d9de1ea8f31a/users/c19f0e6e4d9240fbaeaf155e81b59853/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.520277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9596c1f26040489f57d9de1ea8f31a, user_id=c19f0e6e4d9240fbaeaf155e81b59853, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.523921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.539270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1279/2587] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:49 2022] PUT /identity/v3/projects/bf9596c1f26040489f57d9de1ea8f31a/users/c19f0e6e4d9240fbaeaf155e81b59853/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:49.549504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.549974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.550211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.589105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] MFA Rules not processed for user `c19f0e6e4d9240fbaeaf155e81b59853`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:49.590832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:49.673698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1309/2588] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:49.683755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:49.686046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:49.748714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba481453-c13d-4f28-a0ad-5448012d287e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:49.752391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617730a-e79f-4a05-ab46-2c20c22c337d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:49.753044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617730a-e79f-4a05-ab46-2c20c22c337d service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:49.753328 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617730a-e79f-4a05-ab46-2c20c22c337d service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:49.753603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617730a-e79f-4a05-ab46-2c20c22c337d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:49.763935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617730a-e79f-4a05-ab46-2c20c22c337d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:49.765414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617730a-e79f-4a05-ab46-2c20c22c337d service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:49.775871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1280/2589] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:50.356409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:50.422618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3a140-06ab-41e7-88c7-d9e3fcdc5ca0 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:50.430686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:50.431614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:50.431892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:50.432164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] PATH_INFO: `/v3/users/c0ec565411a5432b930433286ce9c09c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:50.433472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ec565411a5432b930433286ce9c09c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:50.435494 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:50.474542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:50.488676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ec565411a5432b930433286ce9c09c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:50.502384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ec565411a5432b930433286ce9c09c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:50.514665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ec565411a5432b930433286ce9c09c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:50.523810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ec565411a5432b930433286ce9c09c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:50.524689 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ec565411a5432b930433286ce9c09c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:50.534349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1310/2590] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:50 2022] DELETE /identity/v3/users/c0ec565411a5432b930433286ce9c09c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:50.539411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:50.539411 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:50.539971 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:50.565557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:50.567003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:50.582614 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:50.584205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:50.675059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1281/2591] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:50.689009 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:50.695974 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:50.701859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5724b-3457-42ad-8979-4d35bbcdc270 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:50.723139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:50.725489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:50.725882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:50.726385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:50.729179 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:50.729467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:50.746660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:50.749776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:50.783245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1311/2592] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:50.796797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af2421-2f26-4446-a003-eae4962ae583 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:50.801546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:50.802884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:50.803268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:50.803644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:50.816457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:50.819696 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:50.833927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1282/2593] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:21:50.882087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:50.895358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6b93b-1601-4cb6-8c97-effc903573bc service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:50.900052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:50.900779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:50.901332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:50.901455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:50.903832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:50.917138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:50.918669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:50.931841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1312/2594] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:51.579860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.581603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:51.606667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085c8f5-80d6-4aed-9fd9-cae2b2be2240 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.623143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.624969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.625341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.625773 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] PATH_INFO: `/v3/projects/4da70866d1504984ba191ea8c449e0bb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.627626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da70866d1504984ba191ea8c449e0bb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:51.630407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:51.726797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da70866d1504984ba191ea8c449e0bb', 'request_id': 'req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.739349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da70866d1504984ba191ea8c449e0bb', 'request_id': 'req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.747204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da70866d1504984ba191ea8c449e0bb', 'request_id': 'req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.761843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] The token cache is being invalidate because project 4da70866d1504984ba191ea8c449e0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:51.762138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.762811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.764692 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1283/2595] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:51 2022] DELETE /identity/v3/projects/4da70866d1504984ba191ea8c449e0bb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:51.769352 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:51.786978 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c85eaf-639a-4e9d-84e2-0928239266d7 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:51.854180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:51.855421 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:51.855545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:51.855853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] PATH_INFO: `/v3/users/c19f0e6e4d9240fbaeaf155e81b59853` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:51.857144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19f0e6e4d9240fbaeaf155e81b59853)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:51.859135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:51.888271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:51.919118 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19f0e6e4d9240fbaeaf155e81b59853'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.935205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19f0e6e4d9240fbaeaf155e81b59853'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.944032 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19f0e6e4d9240fbaeaf155e81b59853'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.951924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19f0e6e4d9240fbaeaf155e81b59853'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.952754 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-5347f463-42f7-4965-bb07-3f19fada9f45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19f0e6e4d9240fbaeaf155e81b59853'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:51.963826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1313/2596] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:21:51 2022] DELETE /identity/v3/users/c19f0e6e4d9240fbaeaf155e81b59853 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:21:52.289688 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.291281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:52.313350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea88e2-8e9f-4a22-a20d-324f4dbc4a80 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.377044 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.377914 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.378187 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.378453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] PATH_INFO: `/v3/projects/bf9596c1f26040489f57d9de1ea8f31a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.380323 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9596c1f26040489f57d9de1ea8f31a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.382275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.433962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:52.435317 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9596c1f26040489f57d9de1ea8f31a', 'request_id': 'req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:52.443028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9596c1f26040489f57d9de1ea8f31a', 'request_id': 'req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:52.452834 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9596c1f26040489f57d9de1ea8f31a', 'request_id': 'req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:52.466801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] The token cache is being invalidate because project bf9596c1f26040489f57d9de1ea8f31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:21:52.467113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:52.467796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c3f4b411-c9f5-4019-bf7d-4f0a98d3bff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:21:52.469706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1284/2597] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:21:52 2022] DELETE /identity/v3/projects/bf9596c1f26040489f57d9de1ea8f31a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:52.475075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.475389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.475672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.493664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:52.494924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:52.558635 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1314/2598] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:52.630069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.631157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.631444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.651690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:52.657001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:52.672091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1285/2599] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:52.677724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.678509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.679702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:52.708172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf3269-be2c-4287-8e9b-8350a5d92502 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.719903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.719903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.720187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.720686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.721096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:21:52.721402 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.723065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.748420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1315/2600] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:21:52.749410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.773991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e1f9e-1eef-4589-af70-592519f1505a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.781418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9d691-c961-499a-9101-ca16703d4b70 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.782528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.784532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.785345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.785990 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.787523 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.790694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.797117 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.797866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.798216 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.799826 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.802313 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:52.816289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.818365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:52.834037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1286/2601] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:52.847146 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1316/2602] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:52 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:52.851748 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:52.866841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132df0-a82b-4db4-a837-9e6874f4a005 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:52.987391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:52.988143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:52.988429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:52.988702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:52.989805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:52.992037 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.027869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.034526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1287/2603] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:52 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:53.041130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d1f62-de01-482b-be7e-c39d77a71e0a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.047243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.047928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.048264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.048547 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.049459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.051286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.072024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.072723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1317/2604] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:53.088318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40981cfc-3113-4888-b910-48146ef92972 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.095230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66436377-30fc-4cc2-b720-8342633f1247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.096166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436377-30fc-4cc2-b720-8342633f1247 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.096627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436377-30fc-4cc2-b720-8342633f1247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.097090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436377-30fc-4cc2-b720-8342633f1247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.097980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436377-30fc-4cc2-b720-8342633f1247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.099959 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436377-30fc-4cc2-b720-8342633f1247 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.116366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1288/2605] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:53.116750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.128520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43305555-d5a8-40d3-8380-493b758bf86a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.143375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.143375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.143375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.143375 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] PATH_INFO: `/v3/projects/3e110b712c0e42e48ad357f15ee40bc1/users/e072feb4b49c42e0ab73f5295e017725/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.154105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e110b712c0e42e48ad357f15ee40bc1, user_id=e072feb4b49c42e0ab73f5295e017725, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.157193 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.173654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1318/2606] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:53 2022] PUT /identity/v3/projects/3e110b712c0e42e48ad357f15ee40bc1/users/e072feb4b49c42e0ab73f5295e017725/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:21:53.184881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.185148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.185429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.217306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] MFA Rules not processed for user `e072feb4b49c42e0ab73f5295e017725`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:53.218329 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:53.243077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:53.301866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1289/2607] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:53.306504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.351135 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:53.390253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49baf5ab-5acd-4637-9afe-93613eabe318 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.395915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.396716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.397074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.397420 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.398236 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.400270 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.432487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1319/2608] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:53 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:53.434111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.446227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06bfd-ea7b-48ec-a40c-cb6a54d4ff09 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.513074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.513668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.514465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.514814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.515581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.517966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.541542 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1290/2609] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:53 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:53.545673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.556240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7eb24-6a2a-42fc-8896-dda791fcef4e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.561180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.561841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.562121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.562398 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.563165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.564771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.575314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1320/2610] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:21:53.579378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.589970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbb535-955d-4ab8-9a4a-d0fec1301cc5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.594912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc5548-859e-490b-8ac3-6763b9537656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.595570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc5548-859e-490b-8ac3-6763b9537656 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.595852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc5548-859e-490b-8ac3-6763b9537656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.596125 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc5548-859e-490b-8ac3-6763b9537656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.596857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc5548-859e-490b-8ac3-6763b9537656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.598491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc5548-859e-490b-8ac3-6763b9537656 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.608549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1291/2611] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:53.612776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.623454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ee498-2a8c-48a8-8d52-3612e28fb4a1 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.628321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.629211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.629524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.629766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] PATH_INFO: `/v3/projects/c20a40bb0b2546b89179b3557b69664c/users/01cb93125bca4f949e66f6b1636ebc88/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.646241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20a40bb0b2546b89179b3557b69664c, user_id=01cb93125bca4f949e66f6b1636ebc88, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.648975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.664102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1321/2612] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:53 2022] PUT /identity/v3/projects/c20a40bb0b2546b89179b3557b69664c/users/01cb93125bca4f949e66f6b1636ebc88/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:21:53.674501 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.675094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.675209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.705185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] MFA Rules not processed for user `01cb93125bca4f949e66f6b1636ebc88`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:53.706408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:53.775693 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1292/2613] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:53.780357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.851633 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b16f7-c665-420e-b2fb-2285310d94ce admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.856436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.857101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.857377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.857654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.858363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.860307 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.886823 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1322/2614] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:21:53 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:21:53.891559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:53.907333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09241967-c219-4ffd-81a3-6d333d498d36 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:53.941423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:53.972687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:53.973507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:53.973793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:53.974083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:53.974811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:53.976848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:53.997983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1293/2615] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:21:53 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:21:54.002161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:54.013445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1d1b-f696-475e-86ec-b96bed8df88a admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:54.019133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:54.019833 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.020347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.020632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.021377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:54.023031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:54.034280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1323/2616] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:54.038745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:54.049575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f88ed-93b9-4a1c-8fdb-6a69b5a5addf admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:54.055185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:54.056091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.056392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.056638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] PATH_INFO: `/v3/projects/2090bd4dc6274ff8bcf882417a396a1c/users/a0bc4c0740984145842ea14524060550/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.073025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2090bd4dc6274ff8bcf882417a396a1c, user_id=a0bc4c0740984145842ea14524060550, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:54.075709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:54.091973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1294/2617] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:54 2022] PUT /identity/v3/projects/2090bd4dc6274ff8bcf882417a396a1c/users/a0bc4c0740984145842ea14524060550/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:21:54.096232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:54.107174 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51d750-1ce0-4b18-a788-4ebf129dfc45 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:54.165073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:21:54.174837 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e37724-815a-4cf1-86cd-22dff1bdd168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:54.175661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e37724-815a-4cf1-86cd-22dff1bdd168 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.176244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e37724-815a-4cf1-86cd-22dff1bdd168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.177050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e37724-815a-4cf1-86cd-22dff1bdd168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.177929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e37724-815a-4cf1-86cd-22dff1bdd168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:54.179660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e37724-815a-4cf1-86cd-22dff1bdd168 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:54.194325 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1324/2618] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:21:54.197606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:54.208843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a804e4-eb80-41bc-916c-84a61f54238d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:54.214600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:54.215571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.215856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.216131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] PATH_INFO: `/v3/projects/2090bd4dc6274ff8bcf882417a396a1c/users/a0bc4c0740984145842ea14524060550/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.218241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2090bd4dc6274ff8bcf882417a396a1c, user_id=a0bc4c0740984145842ea14524060550, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:54.220943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:54.233437 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1295/2619] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:21:54 2022] PUT /identity/v3/projects/2090bd4dc6274ff8bcf882417a396a1c/users/a0bc4c0740984145842ea14524060550/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:21:54.245339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.245753 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.246106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.278089 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] MFA Rules not processed for user `a0bc4c0740984145842ea14524060550`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:21:54.279434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:54.367766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1325/2620] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:21:54.375493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:54.377399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:54.444181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89f729-3c5e-4c95-9546-8e4c6077a847 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:54.447930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:54.448582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:54.448860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:54.449126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:54.513431 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:54.516246 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:54.529703 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1296/2621] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:56.060301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.062300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:21:56.077468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8035-e00d-4e11-955f-7985dc6cde87 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.081854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.082508 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.082972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.083262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.142297 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.143784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.153601 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1326/2622] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:21:56.740180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:21:56.749488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc227f-e276-4fba-a3b5-254d3abb1d40 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:21:56.753455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c51c1-e94f-48ca-b128-45424d7c071e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:21:56.754281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c51c1-e94f-48ca-b128-45424d7c071e service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:21:56.754476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c51c1-e94f-48ca-b128-45424d7c071e service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:21:56.754810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c51c1-e94f-48ca-b128-45424d7c071e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:21:56.764607 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c51c1-e94f-48ca-b128-45424d7c071e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:21:56.766249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c51c1-e94f-48ca-b128-45424d7c071e service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:21:56.776456 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1297/2623] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:21:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:00.691333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:00.767294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee1623-0489-4d1a-8f5a-6af271805b16 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:00.773072 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:00.773966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:00.774249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:00.774527 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] PATH_INFO: `/v3/users/e072feb4b49c42e0ab73f5295e017725` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:00.775873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e072feb4b49c42e0ab73f5295e017725)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:00.777832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:00.797613 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:00.832874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e072feb4b49c42e0ab73f5295e017725'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:00.845097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e072feb4b49c42e0ab73f5295e017725'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:00.854014 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e072feb4b49c42e0ab73f5295e017725'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:00.862201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e072feb4b49c42e0ab73f5295e017725'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:00.862911 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e072feb4b49c42e0ab73f5295e017725'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:00.869355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1327/2624] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:22:00 2022] DELETE /identity/v3/users/e072feb4b49c42e0ab73f5295e017725 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:00.874609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:00.875146 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:00.875250 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:00.900449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:00.902219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:00.929270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:00.982678 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1298/2625] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:00.995682 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:00.999380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:01.068314 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3065f-3319-466b-bc4f-aa742b784e41 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:01.074226 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:01.075171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.075505 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.075853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.089361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:01.090872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:01.102573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1328/2626] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:01.432684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:01.434294 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:01.450066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba837-b585-48c4-866c-48c2cb2a1c54 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:01.457215 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:01.458749 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.459305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.459756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] PATH_INFO: `/v3/projects/3e110b712c0e42e48ad357f15ee40bc1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.461162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e110b712c0e42e48ad357f15ee40bc1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:01.463818 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:01.525428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e110b712c0e42e48ad357f15ee40bc1', 'request_id': 'req-80fc3975-ed5f-40c3-b881-fed29eb1356b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.532571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e110b712c0e42e48ad357f15ee40bc1', 'request_id': 'req-80fc3975-ed5f-40c3-b881-fed29eb1356b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.540733 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e110b712c0e42e48ad357f15ee40bc1', 'request_id': 'req-80fc3975-ed5f-40c3-b881-fed29eb1356b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.560076 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] The token cache is being invalidate because project 3e110b712c0e42e48ad357f15ee40bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:22:01.560499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.561189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.562873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1299/2627] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:22:01 2022] DELETE /identity/v3/projects/3e110b712c0e42e48ad357f15ee40bc1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:01.567745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:01.585853 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa9d5f-c530-48e6-bb2d-f277c9b8682f service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:01.660814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:01.661863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:01.662157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:01.662503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] PATH_INFO: `/v3/users/01cb93125bca4f949e66f6b1636ebc88` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:01.675667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01cb93125bca4f949e66f6b1636ebc88)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:01.675667 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:01.715581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:01.722191 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01cb93125bca4f949e66f6b1636ebc88'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.740201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01cb93125bca4f949e66f6b1636ebc88'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.751686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01cb93125bca4f949e66f6b1636ebc88'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.759035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01cb93125bca4f949e66f6b1636ebc88'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.759688 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01cb93125bca4f949e66f6b1636ebc88'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:01.765657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1329/2628] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:22:01 2022] DELETE /identity/v3/users/01cb93125bca4f949e66f6b1636ebc88 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:02.069390 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:02.071963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:02.090695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc3975-ed5f-40c3-b881-fed29eb1356b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:02.156008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:02.156865 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:02.157142 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:02.157482 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] PATH_INFO: `/v3/projects/c20a40bb0b2546b89179b3557b69664c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:02.158747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20a40bb0b2546b89179b3557b69664c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:02.160685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:02.209861 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20a40bb0b2546b89179b3557b69664c', 'request_id': 'req-6286c919-23bf-4384-8bea-74281044f300'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.210973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:02.216917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20a40bb0b2546b89179b3557b69664c', 'request_id': 'req-6286c919-23bf-4384-8bea-74281044f300'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.223962 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20a40bb0b2546b89179b3557b69664c', 'request_id': 'req-6286c919-23bf-4384-8bea-74281044f300'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.237571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] The token cache is being invalidate because project c20a40bb0b2546b89179b3557b69664c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:22:02.237859 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.238490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.240085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1300/2629] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:22:02 2022] DELETE /identity/v3/projects/c20a40bb0b2546b89179b3557b69664c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:02.244261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:02.245762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:02.259559 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549133b4-695e-4c12-afc2-ba087e26fa05 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:02.332197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:02.333077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:02.333373 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:02.333643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] PATH_INFO: `/v3/users/a0bc4c0740984145842ea14524060550` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:02.335194 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bc4c0740984145842ea14524060550)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:02.337155 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:02.396288 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bc4c0740984145842ea14524060550'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.409030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bc4c0740984145842ea14524060550'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.418098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bc4c0740984145842ea14524060550'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.424873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bc4c0740984145842ea14524060550'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.425551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d8542e12-25a8-478d-831a-a6a94f12100b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bc4c0740984145842ea14524060550'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.432023 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1330/2630] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:22:02 2022] DELETE /identity/v3/users/a0bc4c0740984145842ea14524060550 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:02.724467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:02.726668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:02.751828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286c919-23bf-4384-8bea-74281044f300 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:02.814354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:02.815310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:02.815594 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:02.815873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] PATH_INFO: `/v3/projects/2090bd4dc6274ff8bcf882417a396a1c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:02.817358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2090bd4dc6274ff8bcf882417a396a1c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:02.820048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:02.871739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2090bd4dc6274ff8bcf882417a396a1c', 'request_id': 'req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.877285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2090bd4dc6274ff8bcf882417a396a1c', 'request_id': 'req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.883407 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2090bd4dc6274ff8bcf882417a396a1c', 'request_id': 'req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.897554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] The token cache is being invalidate because project 2090bd4dc6274ff8bcf882417a396a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:22:02.897840 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.898470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:02.900085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1301/2631] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:22:02 2022] DELETE /identity/v3/projects/2090bd4dc6274ff8bcf882417a396a1c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:04.632629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.633098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.633211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.651383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:04.652625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:04.685661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:04.722127 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1331/2632] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:22:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:04.743605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:04.745204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:04.817126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2447-b737-4ab9-a3b5-fe8bfa3dbbce admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:04.823383 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:04.824032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:04.824308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:04.824575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:04.833683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:04.835507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:04.839471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:04.845833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1302/2633] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:22:05.797531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:05.799022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:05.900204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5929-c24f-405a-b572-ae553a1d2ee5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:05.905677 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:05.906342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:05.906622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:05.906925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:05.908728 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:05.931777 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:05.933423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:05.948783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1332/2634] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:14.851611 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:15.942331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=43 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:18.097292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.098932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.171024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fdce5-0a96-484a-bebf-3ebbb8e69d95 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.174948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.175620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.175904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.176181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.178144 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.192479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.194097 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee555c-841b-44f0-9829-b795076c4aac service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.204814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1303/2635] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:18.529345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:18.531004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.615107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7e7c6-04e9-4f88-b570-e8567b73f4be service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:18.620320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:18.620981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:18.621265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:18.621542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:18.623379 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:18.636197 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:18.637872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:18.649785 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1333/2636] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:22:20.556197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b14d7-1748-4d5a-a824-7d0e0110833d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.556746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b14d7-1748-4d5a-a824-7d0e0110833d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.557047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b14d7-1748-4d5a-a824-7d0e0110833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.581204 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c81b14d7-1748-4d5a-a824-7d0e0110833d None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:20.582554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c81b14d7-1748-4d5a-a824-7d0e0110833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:20.594854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1304/2637] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Jun 24 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:20.608869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:20.620969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d8f35-345f-4152-8e9a-56e0bb3773a1 service placement] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:20.626254 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545d071-c5d4-42e6-99b5-57d2e8afcabe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:20.627130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545d071-c5d4-42e6-99b5-57d2e8afcabe service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:20.627481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545d071-c5d4-42e6-99b5-57d2e8afcabe service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:20.627826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545d071-c5d4-42e6-99b5-57d2e8afcabe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:20.636922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545d071-c5d4-42e6-99b5-57d2e8afcabe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:20.638603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545d071-c5d4-42e6-99b5-57d2e8afcabe service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:20.647567 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1334/2638] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:22:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:22:30.575266 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=29 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:30.644213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=38 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:32.953397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f72855-7cc1-4834-a090-bcc442704796 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.953963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f72855-7cc1-4834-a090-bcc442704796 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.953963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f72855-7cc1-4834-a090-bcc442704796 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.955866 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1305/2639] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:32.962021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db72b5-c292-4fae-bc47-52fab309b157 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:32.963132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db72b5-c292-4fae-bc47-52fab309b157 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:32.963132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db72b5-c292-4fae-bc47-52fab309b157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:32.981546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-c9db72b5-c292-4fae-bc47-52fab309b157 None None] MFA Rules not processed for user `ad007752722c453bafd9e3a8b30226a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:32.982889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c9db72b5-c292-4fae-bc47-52fab309b157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:32.989577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1335/2640] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:33.005295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f72855-7cc1-4834-a090-bcc442704796 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.016945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f72855-7cc1-4834-a090-bcc442704796 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.021352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216b1e7-7007-482e-a5d2-c848610ed3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.022115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216b1e7-7007-482e-a5d2-c848610ed3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.022602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216b1e7-7007-482e-a5d2-c848610ed3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.022820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216b1e7-7007-482e-a5d2-c848610ed3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.034275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216b1e7-7007-482e-a5d2-c848610ed3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.036464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216b1e7-7007-482e-a5d2-c848610ed3e3 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.045311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1306/2641] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:33.181238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1947b-d2c3-4bd6-8e13-0536b2c1daa1 None None] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.181576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1947b-d2c3-4bd6-8e13-0536b2c1daa1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.181908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1947b-d2c3-4bd6-8e13-0536b2c1daa1 None None] PATH_INFO: `/` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.183557 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1336/2642] 10.222.0.64 () {54 vars in 921 bytes} [Fri Jun 24 10:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:22:33.189372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa01d-08ce-41ef-b056-a91421af94e0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.189670 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa01d-08ce-41ef-b056-a91421af94e0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.189949 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa01d-08ce-41ef-b056-a91421af94e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.208966 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dadaa01d-08ce-41ef-b056-a91421af94e0 None None] MFA Rules not processed for user `ac88daa9b11b4e13b89f3f981686ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:33.210174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dadaa01d-08ce-41ef-b056-a91421af94e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:33.220395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1307/2643] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Jun 24 10:22:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:33.233088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1947b-d2c3-4bd6-8e13-0536b2c1daa1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:33.235000 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f0c1947b-d2c3-4bd6-8e13-0536b2c1daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:33.249331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1947b-d2c3-4bd6-8e13-0536b2c1daa1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:33.253158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:33.253790 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:33.254076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:33.254378 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:33.265820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:33.267321 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:33.289460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1337/2644] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:35.289534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.290021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.290126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.309785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:35.311095 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:35.319756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1308/2645] 10.222.0.64 () {58 vars in 967 bytes} [Fri Jun 24 10:22:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:35.332220 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:35.347267 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3f78-59c0-44ca-a49d-5dd20add3822 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:35.353280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:35.353918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:35.354772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:35.354772 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:35.363934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:35.365537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:35.374231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1338/2646] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:22:45.303906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=12 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:45.370890 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=15 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:48.860308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:48.870494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3b24d-676c-48fe-856f-8d45bea7ca91 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:48.874008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:48.874646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:48.875065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:48.875310 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:48.877093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:48.945652 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:48.947190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:48.957602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1309/2647] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:52.368762 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:52.392629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf856594-0631-47b4-a36c-0fc2e412ac53 service swift] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:52.401799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:52.402821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:52.403237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:52.403612 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:52.406317 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:52.444382 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:52.445780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:52.447362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:52.449434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:52.467190 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1339/2648] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:52.477106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63259f-0021-42fb-9adf-6f2ddf81ed8e service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:52.483552 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:52.484442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:52.484721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:52.484995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] PATH_INFO: `/v3/users/6ca14b36fc754fe9b24abdfecf0f64c0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:52.486225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca14b36fc754fe9b24abdfecf0f64c0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:52.488222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:52.537595 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca14b36fc754fe9b24abdfecf0f64c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.549906 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca14b36fc754fe9b24abdfecf0f64c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.557837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca14b36fc754fe9b24abdfecf0f64c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.565889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca14b36fc754fe9b24abdfecf0f64c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.566561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-903756a5-e18c-40c7-8884-392765f1040d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca14b36fc754fe9b24abdfecf0f64c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.573118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1310/2649] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:22:52 2022] DELETE /identity/v3/users/6ca14b36fc754fe9b24abdfecf0f64c0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:52.756589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:52.767955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5967-0fae-42eb-a0a4-b86a1ba9853a service placement] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:52.828163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:52.829043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:52.829319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:52.829605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] PATH_INFO: `/v3/projects/a33a4f77f0ae4dc09d4f8c29babf3b03` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:52.830846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33a4f77f0ae4dc09d4f8c29babf3b03)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:52.832774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:52.881130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33a4f77f0ae4dc09d4f8c29babf3b03', 'request_id': 'req-34d76e9c-1123-4bb2-bff1-e58570c6221d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.887696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33a4f77f0ae4dc09d4f8c29babf3b03', 'request_id': 'req-34d76e9c-1123-4bb2-bff1-e58570c6221d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.896067 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33a4f77f0ae4dc09d4f8c29babf3b03', 'request_id': 'req-34d76e9c-1123-4bb2-bff1-e58570c6221d'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.906576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] The token cache is being invalidate because project a33a4f77f0ae4dc09d4f8c29babf3b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:22:52.906893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.907531 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-34d76e9c-1123-4bb2-bff1-e58570c6221d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:22:52.909384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1340/2650] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:22:52 2022] DELETE /identity/v3/projects/a33a4f77f0ae4dc09d4f8c29babf3b03 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:22:52.914172 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:52.914605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:52.914919 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:52.934596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:52.935877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:52.999549 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1311/2651] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:53.007573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.007848 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.008141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.027281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:53.028512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:53.036934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1341/2652] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:22:53.041557 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.052366 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01b6ff-a86c-4aa6-bb6b-a69bf3b3af53 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.057812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.058469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.058735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.059059 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.059795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:22:53.060099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.061690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.063316 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:53.079339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1312/2653] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:22:53.080561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.092913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ee7b9-9bf9-463f-8b3a-e959e81fd183 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.099205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.099933 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.102766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.102766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.102766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.103201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.146417 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1342/2654] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:22:53 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:22:53.147472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.158335 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275c4e4-4c0e-4e8d-9c52-ca575f4f697e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.219013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.219697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.219981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.220255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.220970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.222935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.244283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1313/2655] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:22:53 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:22:53.249356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.261105 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3181164-f28f-41af-ad3a-a214d408db0e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.266841 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.267518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.267801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.268076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.268845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.270453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.281285 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1343/2656] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:22:53.285656 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.296312 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29163571-0096-4a66-b087-a53b2544667c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.301271 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388106-620e-492b-b5d3-583ce6aebcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.302049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388106-620e-492b-b5d3-583ce6aebcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.302372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388106-620e-492b-b5d3-583ce6aebcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.302725 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388106-620e-492b-b5d3-583ce6aebcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.303523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388106-620e-492b-b5d3-583ce6aebcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.305225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388106-620e-492b-b5d3-583ce6aebcb7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.316304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1314/2657] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:22:53.321518 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.332540 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ab32b-8925-4145-857f-fe357b863931 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.338546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.339480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.339767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.340048 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] PATH_INFO: `/v3/projects/13b216e0dd4e46e2a2f4b275666118f6/users/f96ee29f9c5b4651825da16d0520a0e8/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.356502 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b216e0dd4e46e2a2f4b275666118f6, user_id=f96ee29f9c5b4651825da16d0520a0e8, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.359264 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.373746 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1344/2658] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:22:53 2022] PUT /identity/v3/projects/13b216e0dd4e46e2a2f4b275666118f6/users/f96ee29f9c5b4651825da16d0520a0e8/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:22:53.379196 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1786fc-8831-479a-a236-8a34f3aade37 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.379481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1786fc-8831-479a-a236-8a34f3aade37 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.379761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1786fc-8831-479a-a236-8a34f3aade37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.398126 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2f1786fc-8831-479a-a236-8a34f3aade37 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:53.399668 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2f1786fc-8831-479a-a236-8a34f3aade37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:53.459714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1315/2659] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:53.470459 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:53.472477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:53.479721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:53.551896 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111a092-ef79-4bef-a371-81c785bbb463 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:53.555906 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:53.556564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:53.556854 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:53.557130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:53.567345 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:53.568819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:53.579258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1345/2660] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:57.944956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fab198-da17-4848-b567-e500cfa75b6c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:57.944956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fab198-da17-4848-b567-e500cfa75b6c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:57.945691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fab198-da17-4848-b567-e500cfa75b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:57.985747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-79fab198-da17-4848-b567-e500cfa75b6c None None] MFA Rules not processed for user `f96ee29f9c5b4651825da16d0520a0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:57.987034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-79fab198-da17-4848-b567-e500cfa75b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:58.072620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1316/2661] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:22:58.088418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:22:58.090074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:58.194157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8b08f-0ef9-4cf6-95fb-9cfa449ca253 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:22:58.203227 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80656c1a-fa06-468f-8a52-ee8ad2799fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:22:58.204234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80656c1a-fa06-468f-8a52-ee8ad2799fff service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:58.204636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80656c1a-fa06-468f-8a52-ee8ad2799fff service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:58.205069 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80656c1a-fa06-468f-8a52-ee8ad2799fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:58.216134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80656c1a-fa06-468f-8a52-ee8ad2799fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:22:58.218042 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80656c1a-fa06-468f-8a52-ee8ad2799fff service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:22:58.228770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1346/2662] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:22:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:22:59.693003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:22:59.693601 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:22:59.693983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:22:59.701814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:22:59.716463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] MFA Rules not processed for user `f96ee29f9c5b4651825da16d0520a0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:22:59.718534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:22:59.734642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1317/2663] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:22:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:08.223661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=58 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:09.718561 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:16.432272 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.432802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.432802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.450915 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] MFA Rules not processed for user `22d7019a89fb45c6afd1076f1ea120af`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:16.452128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:16.511591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1347/2664] 10.222.0.64 () {58 vars in 967 bytes} [Fri Jun 24 10:23:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:16.535871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:16.537408 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:16.628008 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ccc04-6bd1-47cb-93be-38d283bcc6b9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:16.635112 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19fd34c5b9e34440b7c3752ff1cdf4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:16.635786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:16.636069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:16.636341 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:16.645876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:16.647714 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:16.657138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1318/2665] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Jun 24 10:23:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 10:23:19.319063 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:19.320621 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:19.333771 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40be26a-b4e2-44d5-a1a4-9ab7e1fab6f0 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:19.338324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:19.339101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:19.339389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:19.339669 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:19.341581 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:19.354882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:19.356342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:19.366918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1348/2666] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:23:20.693237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:20.696970 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:20.857662 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b478fb-5bda-4a0a-8521-54f1246ce70e service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:20.867033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:20.868397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:20.869072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:20.869706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:20.873067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:20.899921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:20.901658 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:20.916286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1319/2667] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:24.292548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.295387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:24.309301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591681d0-ffb1-4d37-9b4e-24606b387dec service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.318077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.319128 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.319407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.319678 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] PATH_INFO: `/v3/users/0a66b13572904a309d3180afc7278070` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.321670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a66b13572904a309d3180afc7278070)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.323907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.390619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a66b13572904a309d3180afc7278070'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.411200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a66b13572904a309d3180afc7278070'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.420445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a66b13572904a309d3180afc7278070'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.427850 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a66b13572904a309d3180afc7278070'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.428543 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-4a44a2f2-b4a2-48e9-aeb8-42abb96f0b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a66b13572904a309d3180afc7278070'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.435405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1349/2668] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:24 2022] DELETE /identity/v3/users/0a66b13572904a309d3180afc7278070 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:24.638077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:24.650574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7debb-ebcf-47b0-9cee-b1102c1e3f63 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:24.749524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:24.750423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.750698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.751005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] PATH_INFO: `/v3/projects/74111042535d44c8a52a4656eb37fa23` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.752217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] RBAC: Authorizing `identity:delete_project(project_id=74111042535d44c8a52a4656eb37fa23)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:24.754113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:24.781947 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=100 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:24.812054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74111042535d44c8a52a4656eb37fa23', 'request_id': 'req-150e9cee-acde-43ee-a448-88e37b46c25b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.817987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74111042535d44c8a52a4656eb37fa23', 'request_id': 'req-150e9cee-acde-43ee-a448-88e37b46c25b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.824946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74111042535d44c8a52a4656eb37fa23', 'request_id': 'req-150e9cee-acde-43ee-a448-88e37b46c25b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.840712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] The token cache is being invalidate because project 74111042535d44c8a52a4656eb37fa23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:24.841074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.841683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-150e9cee-acde-43ee-a448-88e37b46c25b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:24.843779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1320/2669] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:24 2022] DELETE /identity/v3/projects/74111042535d44c8a52a4656eb37fa23 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:24.849277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:24.851175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:24.851900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:24.885300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:24.887425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.002004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1350/2670] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:25.010034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.010319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.011833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.029232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:25.030546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.046614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1321/2671] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:25.050838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.057710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.063208 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23956d90-bdba-4507-baad-0e02a77f276b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.070869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.071563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.071844 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.072117 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.073157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:25.073589 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.077157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.100629 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1351/2672] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 10:23:25.100967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.116452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8ec2-6dfe-4a82-82f1-a6d773f60aa2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.125489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.126177 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.126462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.126752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.127479 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.129432 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.171094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1322/2673] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/projects => generated 405 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:25.174868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.188153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55e563-0a49-4e72-a0fb-d5973f2aeff4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.257083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.258108 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.258467 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.259005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.259787 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.261845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.297720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1352/2674] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:25.298908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.310349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a899c-e715-4c08-9061-2f1b75e2f758 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.318085 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.318562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.318888 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.319162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.319907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.323248 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.336875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1323/2675] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:25.341845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.352877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbfd9b-a7a5-4747-993f-b847b7a673f3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.357632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.358311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.358594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.359097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.359999 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.361620 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.374755 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1353/2676] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:25.376709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.386975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4ec54-672e-4603-896d-83fb49f7dce6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.391690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd9a9d-47ba-4b8e-8d6d-dd9b5c5c63da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.392545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd9a9d-47ba-4b8e-8d6d-dd9b5c5c63da admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.392820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd9a9d-47ba-4b8e-8d6d-dd9b5c5c63da admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.393122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd9a9d-47ba-4b8e-8d6d-dd9b5c5c63da admin admin] PATH_INFO: `/v3/projects/a8123631baba4913875b2e74352f7346/users/b1239351a30142c2864bd39f929e35fb/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.410136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd9a9d-47ba-4b8e-8d6d-dd9b5c5c63da admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8123631baba4913875b2e74352f7346, user_id=b1239351a30142c2864bd39f929e35fb, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.412869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd9a9d-47ba-4b8e-8d6d-dd9b5c5c63da admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.423265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.425428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.430056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1324/2677] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:25 2022] PUT /identity/v3/projects/a8123631baba4913875b2e74352f7346/users/b1239351a30142c2864bd39f929e35fb/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:25.444488 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.444788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.445106 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.446739 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891664-b654-4fb1-8e36-4c8a4c633770 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.473251 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] MFA Rules not processed for user `b1239351a30142c2864bd39f929e35fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:25.474484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.506599 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.507654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.507946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.508218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] PATH_INFO: `/v3/users/ef90ff4734c249ab965ebf920270b20b` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.509687 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef90ff4734c249ab965ebf920270b20b)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.511717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.544394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1325/2678] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:25.549123 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.560629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05563f89-06b0-4bd0-93b0-9e0121ebb88b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.565618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef90ff4734c249ab965ebf920270b20b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.566311 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.567041 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.567463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.567743 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.568450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.570469 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.578012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef90ff4734c249ab965ebf920270b20b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.586680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef90ff4734c249ab965ebf920270b20b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.589156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.598145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef90ff4734c249ab965ebf920270b20b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.599309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef90ff4734c249ab965ebf920270b20b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.604400 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1326/2679] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:23:25.605881 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.607075 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1354/2680] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:25 2022] DELETE /identity/v3/users/ef90ff4734c249ab965ebf920270b20b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:25.607732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.622981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:25.680322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbd367-8d7e-401e-84e7-a39bdfe69089 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.746270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.747007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.747290 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.747562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.748299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.750372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.774956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1327/2681] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:23:25 2022] POST /identity/v3/users => generated 309 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:23:25.780081 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.793575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32776701-e810-474b-8260-9e8f923267e0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.800573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.801273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.801553 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.801826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.802573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.804403 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.808006 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.810046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.819579 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1355/2682] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:25.821020 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.830576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e91464-032b-4793-a3ac-67d31cb37a84 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.832147 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1431e-052e-4a4e-8623-fcb2862cffe6 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.835926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.836903 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.837038 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:25.837771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.837923 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] PATH_INFO: `/v3/projects/34f04f4ab2294464bdc83acecb4d9a79` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.838279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:25.838562 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:25.838889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] PATH_INFO: `/v3/projects/7ff41fc0f55942ed89f8045252c3826c/users/f0d1c13f839041b4b82a33a0e32e8739/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:25.839532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f04f4ab2294464bdc83acecb4d9a79)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.841484 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.856929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff41fc0f55942ed89f8045252c3826c, user_id=f0d1c13f839041b4b82a33a0e32e8739, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:25.859654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:25.875005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1356/2683] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:25 2022] PUT /identity/v3/projects/7ff41fc0f55942ed89f8045252c3826c/users/f0d1c13f839041b4b82a33a0e32e8739/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:25.879029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.891079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f04f4ab2294464bdc83acecb4d9a79', 'request_id': 'req-417171da-1026-408b-8d22-1331f5f3ff5c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.896930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f04f4ab2294464bdc83acecb4d9a79', 'request_id': 'req-417171da-1026-408b-8d22-1331f5f3ff5c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.904136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f04f4ab2294464bdc83acecb4d9a79', 'request_id': 'req-417171da-1026-408b-8d22-1331f5f3ff5c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.920165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] The token cache is being invalidate because project 34f04f4ab2294464bdc83acecb4d9a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:25.920454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.921715 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:25.923795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1328/2684] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:25 2022] DELETE /identity/v3/projects/34f04f4ab2294464bdc83acecb4d9a79 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:25.927790 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:25.929753 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:25.952130 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bfd4d-160b-417b-b49d-c17d8878a298 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:25.969606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.010510 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417171da-1026-408b-8d22-1331f5f3ff5c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.015340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.016236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.016517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.016797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] PATH_INFO: `/v3/users/ace547a802324ef99b14ae40b1fe1370` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.017132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.017819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.018022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace547a802324ef99b14ae40b1fe1370)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.018139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.018369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.019145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.020002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.020749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.031986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1357/2685] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:26.035969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.040424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.055320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc432c-dbd2-4a4e-baf5-5c1e1afb21ff admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.059255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb7bf7-ca46-4242-96de-69c79cad8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.060165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb7bf7-ca46-4242-96de-69c79cad8134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.060450 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb7bf7-ca46-4242-96de-69c79cad8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.060724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb7bf7-ca46-4242-96de-69c79cad8134 admin admin] PATH_INFO: `/v3/projects/7ff41fc0f55942ed89f8045252c3826c/users/f0d1c13f839041b4b82a33a0e32e8739/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.062710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb7bf7-ca46-4242-96de-69c79cad8134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff41fc0f55942ed89f8045252c3826c, user_id=f0d1c13f839041b4b82a33a0e32e8739, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.065445 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb7bf7-ca46-4242-96de-69c79cad8134 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.073337 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace547a802324ef99b14ae40b1fe1370'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.076593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1358/2686] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:26 2022] PUT /identity/v3/projects/7ff41fc0f55942ed89f8045252c3826c/users/f0d1c13f839041b4b82a33a0e32e8739/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:26.085191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace547a802324ef99b14ae40b1fe1370'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.087486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.087776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.088050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.094827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace547a802324ef99b14ae40b1fe1370'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.103520 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace547a802324ef99b14ae40b1fe1370'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.104260 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace547a802324ef99b14ae40b1fe1370'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.110330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1329/2687] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:25 2022] DELETE /identity/v3/users/ace547a802324ef99b14ae40b1fe1370 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:26.124129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] MFA Rules not processed for user `f0d1c13f839041b4b82a33a0e32e8739`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:26.125354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.165210 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.206268 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1359/2688] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:26.217801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.219351 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.283681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327644b9-f750-4f2b-afe4-34bca19d917f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.287864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.288515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.288791 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.289090 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.293311 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.295037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.300162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.301943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.313389 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1330/2689] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:26 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:26.382112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d208d0f-3a40-4154-8335-51078040bd64 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.387102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.387989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.388273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.388552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] PATH_INFO: `/v3/projects/05a745d261eb464a95f6f1437e87ad8d` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.389743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a745d261eb464a95f6f1437e87ad8d)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.391692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.444026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a745d261eb464a95f6f1437e87ad8d', 'request_id': 'req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.451207 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a745d261eb464a95f6f1437e87ad8d', 'request_id': 'req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.455422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.457604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.460123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a745d261eb464a95f6f1437e87ad8d', 'request_id': 'req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.471992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288d490-57dc-4a96-a70c-8bba9fe66096 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.472773 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] The token cache is being invalidate because project 05a745d261eb464a95f6f1437e87ad8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:26.473071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.473829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-009e1ae6-ccd2-4d29-9ec6-9f97162cd58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.475692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1360/2690] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:26 2022] DELETE /identity/v3/projects/05a745d261eb464a95f6f1437e87ad8d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:26.528886 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.539024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.540547 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.541185 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.541808 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] PATH_INFO: `/v3/users/b1239351a30142c2864bd39f929e35fb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.543943 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1239351a30142c2864bd39f929e35fb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.545967 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.604028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1239351a30142c2864bd39f929e35fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.617225 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1239351a30142c2864bd39f929e35fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.626794 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1239351a30142c2864bd39f929e35fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.635330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1239351a30142c2864bd39f929e35fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.636094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1239351a30142c2864bd39f929e35fb'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:26.642801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1331/2691] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:26 2022] DELETE /identity/v3/users/b1239351a30142c2864bd39f929e35fb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:26.647826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.648198 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.648388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.670219 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:26.671514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.748351 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:26.758436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1361/2692] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:26.770066 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:26.771690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:26.836544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bb15a-b10e-4f47-b506-b0533fd1f924 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:26.840880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:26.841517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:26.841812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:26.842061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:26.851363 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:26.853286 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:26.863016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1332/2693] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:27.156384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.157944 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:27.174464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb75f-87a6-4b08-a63a-a6787a7f1583 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.180095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.181410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.181686 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.182050 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] PATH_INFO: `/v3/projects/a8123631baba4913875b2e74352f7346` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.183347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8123631baba4913875b2e74352f7346)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:27.185240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:27.236873 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8123631baba4913875b2e74352f7346', 'request_id': 'req-f142c929-7fb1-40be-a356-8b4972fc447e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.242977 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8123631baba4913875b2e74352f7346', 'request_id': 'req-f142c929-7fb1-40be-a356-8b4972fc447e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.253281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8123631baba4913875b2e74352f7346', 'request_id': 'req-f142c929-7fb1-40be-a356-8b4972fc447e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.264195 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] The token cache is being invalidate because project a8123631baba4913875b2e74352f7346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:27.264481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.265095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.266681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1362/2694] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:27 2022] DELETE /identity/v3/projects/a8123631baba4913875b2e74352f7346 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:27.270796 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.281071 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6169f6-678b-424a-a159-139253969709 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.340118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.341030 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.341308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.341578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] PATH_INFO: `/v3/users/f0d1c13f839041b4b82a33a0e32e8739` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.342842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d1c13f839041b4b82a33a0e32e8739)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:27.344873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:27.366321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:27.395288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d1c13f839041b4b82a33a0e32e8739'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.406895 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d1c13f839041b4b82a33a0e32e8739'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.415493 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d1c13f839041b4b82a33a0e32e8739'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.422264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d1c13f839041b4b82a33a0e32e8739'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.423057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-123804e8-54b4-411e-92cc-615dd833a204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d1c13f839041b4b82a33a0e32e8739'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.428880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1333/2695] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:27 2022] DELETE /identity/v3/users/f0d1c13f839041b4b82a33a0e32e8739 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:27.709576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:27.711371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:27.726533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c929-7fb1-40be-a356-8b4972fc447e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:27.784561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:27.785366 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.785646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.785926 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] PATH_INFO: `/v3/projects/7ff41fc0f55942ed89f8045252c3826c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.787202 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff41fc0f55942ed89f8045252c3826c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:27.789160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:27.838515 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff41fc0f55942ed89f8045252c3826c', 'request_id': 'req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.845329 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff41fc0f55942ed89f8045252c3826c', 'request_id': 'req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.854163 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff41fc0f55942ed89f8045252c3826c', 'request_id': 'req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.864664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] The token cache is being invalidate because project 7ff41fc0f55942ed89f8045252c3826c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:27.864953 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.865576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e0ceb321-9e09-4be7-ab8d-ff2d452fb06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:27.867184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1363/2696] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:27 2022] DELETE /identity/v3/projects/7ff41fc0f55942ed89f8045252c3826c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:27.872179 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.872674 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.872887 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.891590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:27.892783 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:27.963371 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1334/2697] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:27.971552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:27.971843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:27.972126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:27.992156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:27.993383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:28.002555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1364/2698] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:28.008368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.019084 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e6133-3d9f-4bc6-9f23-823d0ae8d61c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.025302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.025958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.026238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.026513 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.027284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:28.027588 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.029203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.041863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1335/2699] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:28.046196 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.056865 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c000c19-6edc-4fc1-9933-fdedb382ff03 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.061838 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.062504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.062818 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.063098 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.063801 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.065724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.069083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.095849 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1365/2700] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:28.100054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.110587 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb53d5-1fa3-4004-a154-c212a23bfbfb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.171189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.171863 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.172147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.172426 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.173131 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.175133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.198120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1336/2701] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:23:28.200087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.210548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f0478-bc7c-43af-81ee-6522ba026eda admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.215745 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.216405 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.216690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.216966 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.217706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.219358 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.229447 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1366/2702] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:28.233554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.243905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e018b-122f-4126-9021-a29f90d67c8f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.248641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2be9f9-e348-4af1-98e0-1a616dcf4dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.249289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2be9f9-e348-4af1-98e0-1a616dcf4dde admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.249585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2be9f9-e348-4af1-98e0-1a616dcf4dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.249858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2be9f9-e348-4af1-98e0-1a616dcf4dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.250600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2be9f9-e348-4af1-98e0-1a616dcf4dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.252254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2be9f9-e348-4af1-98e0-1a616dcf4dde admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.262618 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1337/2703] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:28.266868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.278073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294b955-a988-4cc6-a643-ccf66ebe5a65 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.283880 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.284802 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.285083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.285356 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] PATH_INFO: `/v3/projects/08bff9bc9cd742bf9cd4b20ce8568ed9/users/5d30419c6a5b49f1a07da368b5dbf46e/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.300831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bff9bc9cd742bf9cd4b20ce8568ed9, user_id=5d30419c6a5b49f1a07da368b5dbf46e, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.303524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.321133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1367/2704] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:28 2022] PUT /identity/v3/projects/08bff9bc9cd742bf9cd4b20ce8568ed9/users/5d30419c6a5b49f1a07da368b5dbf46e/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:28.332680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.333237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.333648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.340798 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.365995 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] MFA Rules not processed for user `5d30419c6a5b49f1a07da368b5dbf46e`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:28.367394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:28.435423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1338/2705] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:23:28.440087 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.504352 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7fd5b-af87-4bf7-9d08-3c31796cb517 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.510241 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.510942 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.511231 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.511512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.512218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.514151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.542365 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1368/2706] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:28.546534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.557763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d3dc1-3711-4622-acfd-f722aa613413 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.616751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.617578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.618145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.618418 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.619151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.621127 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.643879 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1339/2707] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:28.647495 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.659384 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee1c2-b671-4968-a5a4-e9a127b82758 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.665497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.666164 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.666446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.666731 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.667499 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.669109 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.680033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1369/2708] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:23:28.684486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.694958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712306ec-b132-4836-aed2-c2ffd0ef633a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.700550 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.701447 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.701727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.702002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] PATH_INFO: `/v3/projects/ee9c21e72132438b88e8a81b29cecdcb/users/7656904cd754495d9f29e4a49498495d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.718885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9c21e72132438b88e8a81b29cecdcb, user_id=7656904cd754495d9f29e4a49498495d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.721569 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.736710 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1340/2709] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:28 2022] PUT /identity/v3/projects/ee9c21e72132438b88e8a81b29cecdcb/users/7656904cd754495d9f29e4a49498495d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:28.740916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.751659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba28c3-4923-4ab6-919a-1fda990cc3f6 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.811869 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b1a6f-1079-4bc7-be12-03473034b868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.812451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b1a6f-1079-4bc7-be12-03473034b868 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.812737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b1a6f-1079-4bc7-be12-03473034b868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.813008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b1a6f-1079-4bc7-be12-03473034b868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.813735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b1a6f-1079-4bc7-be12-03473034b868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.815369 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b1a6f-1079-4bc7-be12-03473034b868 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.825694 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1370/2710] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:28.830161 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:28.845145 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de7de0-97f6-40bf-bfc1-218b3d103c15 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:28.852823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:28.853705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.853983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.854255 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] PATH_INFO: `/v3/projects/ee9c21e72132438b88e8a81b29cecdcb/users/7656904cd754495d9f29e4a49498495d/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.856395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9c21e72132438b88e8a81b29cecdcb, user_id=7656904cd754495d9f29e4a49498495d, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:28.859063 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:28.870051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1341/2711] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:28 2022] PUT /identity/v3/projects/ee9c21e72132438b88e8a81b29cecdcb/users/7656904cd754495d9f29e4a49498495d/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:28.880332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:28.880625 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:28.880902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:28.909989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] MFA Rules not processed for user `7656904cd754495d9f29e4a49498495d`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:28.911243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:28.921610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:28.994668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1371/2712] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:29.010065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.012073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:29.063932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:29.082442 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd596f-965e-48bc-86dc-d6e06e5bb3dd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.088585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.089230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.089556 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.089827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.100166 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.101686 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.111745 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1342/2713] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:29.190723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:29.202034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e1a00-8707-4d2e-807e-b06dfc8957eb None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:29.206286 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439d318-c76d-452e-8436-08f807ed8c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:29.206923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439d318-c76d-452e-8436-08f807ed8c8d service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:29.207205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439d318-c76d-452e-8436-08f807ed8c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:29.207517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439d318-c76d-452e-8436-08f807ed8c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:29.267706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439d318-c76d-452e-8436-08f807ed8c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:29.269153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439d318-c76d-452e-8436-08f807ed8c8d service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:29.280514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1372/2714] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:32.073632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.140797 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca3a48-c9eb-4937-81b4-e8553372630b service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.145609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.146540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.146853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.147151 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] PATH_INFO: `/v3/users/5d30419c6a5b49f1a07da368b5dbf46e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.148452 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d30419c6a5b49f1a07da368b5dbf46e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.150391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.196812 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d30419c6a5b49f1a07da368b5dbf46e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.208807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d30419c6a5b49f1a07da368b5dbf46e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.217506 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d30419c6a5b49f1a07da368b5dbf46e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.224681 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d30419c6a5b49f1a07da368b5dbf46e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.225306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d30419c6a5b49f1a07da368b5dbf46e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.231451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1343/2715] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:32 2022] DELETE /identity/v3/users/5d30419c6a5b49f1a07da368b5dbf46e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:32.236414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.236940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.236940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.254727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:32.255963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:32.322597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1373/2716] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:23:32.332985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.334555 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:32.407302 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79617b-4940-40e1-ba1e-8d912933067f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.412134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.412784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.413062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.413373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.423989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.425481 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.435804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1344/2717] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:32.733576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.735292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:32.750144 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c06ed-8319-43ed-a7a9-c41b3e575de4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.756158 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.757222 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.757542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.757862 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] PATH_INFO: `/v3/projects/08bff9bc9cd742bf9cd4b20ce8568ed9` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.759446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bff9bc9cd742bf9cd4b20ce8568ed9)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.761742 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.805868 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:32.814049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bff9bc9cd742bf9cd4b20ce8568ed9', 'request_id': 'req-87dc1fd3-0731-4dd0-b38c-929f78f87b21'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.820529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bff9bc9cd742bf9cd4b20ce8568ed9', 'request_id': 'req-87dc1fd3-0731-4dd0-b38c-929f78f87b21'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.829416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bff9bc9cd742bf9cd4b20ce8568ed9', 'request_id': 'req-87dc1fd3-0731-4dd0-b38c-929f78f87b21'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.841152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] The token cache is being invalidate because project 08bff9bc9cd742bf9cd4b20ce8568ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:32.841460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.842083 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.843836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1374/2718] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:32 2022] DELETE /identity/v3/projects/08bff9bc9cd742bf9cd4b20ce8568ed9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:32.849676 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:32.860517 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001be8-1858-4c18-85cb-7abb582e6ae6 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:32.865604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:32.919193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:32.920069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:32.920345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:32.920774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] PATH_INFO: `/v3/users/7656904cd754495d9f29e4a49498495d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:32.921921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7656904cd754495d9f29e4a49498495d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:32.924042 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:32.973805 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7656904cd754495d9f29e4a49498495d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.986817 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7656904cd754495d9f29e4a49498495d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:32.997000 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7656904cd754495d9f29e4a49498495d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.008726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7656904cd754495d9f29e4a49498495d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.009754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e8a4e51-357f-481a-9257-ce9e0102d577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7656904cd754495d9f29e4a49498495d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.017309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1345/2719] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:23:32 2022] DELETE /identity/v3/users/7656904cd754495d9f29e4a49498495d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:33.301349 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.302889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:33.316512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1fd3-0731-4dd0-b38c-929f78f87b21 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.375009 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.375892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.376175 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.376451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] PATH_INFO: `/v3/projects/ee9c21e72132438b88e8a81b29cecdcb` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.377664 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9c21e72132438b88e8a81b29cecdcb)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.379634 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.431213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9c21e72132438b88e8a81b29cecdcb', 'request_id': 'req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.437913 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9c21e72132438b88e8a81b29cecdcb', 'request_id': 'req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.446035 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9c21e72132438b88e8a81b29cecdcb', 'request_id': 'req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.457177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] The token cache is being invalidate because project ee9c21e72132438b88e8a81b29cecdcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:23:33.457464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.458141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-881a9d6a-dab0-4036-a2e6-75a9c6c3ecfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:23:33.459761 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1375/2720] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:23:33 2022] DELETE /identity/v3/projects/ee9c21e72132438b88e8a81b29cecdcb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:23:33.465138 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.465754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.465754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.484677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:33.485880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:33.562629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1346/2721] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:33.571925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.572209 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.572488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.591791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:33.593019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:33.601482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1376/2722] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:33.605924 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.616073 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83622bb-6363-44b6-91dc-a473648c76b3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.621284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.621869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.622157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.622420 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.623171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:23:33.623453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.625121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.636950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1347/2723] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:23:33.641748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.653084 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418430-26ce-4547-a289-a141bf9b6075 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.658382 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.659106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.659392 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.659666 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.660361 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.662257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.695045 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1377/2724] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:33.696330 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.707111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169aa346-89a2-484d-a183-f6f8ba20166b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.722322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.774352 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.775035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.775314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.775585 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.776275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.778265 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.800835 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1348/2725] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:23:33.806253 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.818798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0226f27-b84f-41ae-8b67-6acdb626dc60 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.825371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.826125 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.826425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.826657 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.827460 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.829074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.841760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1378/2726] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:33.846065 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.856814 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80f2ee-9b3e-4394-aef4-c809d45b5fdc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.862155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5c366-609e-48ea-9cb4-bef2418e017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.862843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5c366-609e-48ea-9cb4-bef2418e017d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.863120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5c366-609e-48ea-9cb4-bef2418e017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.863379 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5c366-609e-48ea-9cb4-bef2418e017d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.864107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5c366-609e-48ea-9cb4-bef2418e017d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.865705 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5c366-609e-48ea-9cb4-bef2418e017d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.876541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1349/2727] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:23:33.880511 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:33.890940 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea78d1-1699-4346-8309-444c05e5a54e admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:33.895981 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:33.896863 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.897145 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.897438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] PATH_INFO: `/v3/projects/28bd315147274a3ca9174fc45a4fe861/users/bc9128f69a794f55b677500efbe87bf2/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.912836 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bd315147274a3ca9174fc45a4fe861, user_id=bc9128f69a794f55b677500efbe87bf2, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:33.915521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:33.917706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:33.937574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1379/2728] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:23:33 2022] PUT /identity/v3/projects/28bd315147274a3ca9174fc45a4fe861/users/bc9128f69a794f55b677500efbe87bf2/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:23:33.949899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1999cd8-c2aa-4486-8f6e-df3e3736caf5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:33.950685 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1999cd8-c2aa-4486-8f6e-df3e3736caf5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:33.951033 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1999cd8-c2aa-4486-8f6e-df3e3736caf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:33.979130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b1999cd8-c2aa-4486-8f6e-df3e3736caf5 None None] MFA Rules not processed for user `bc9128f69a794f55b677500efbe87bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:23:33.980334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b1999cd8-c2aa-4486-8f6e-df3e3736caf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:34.054907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1350/2729] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:23:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:23:34.066857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:23:34.068674 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:23:34.134161 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f9f5-7ee9-4066-8ff0-f5ead711d196 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:23:34.138676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb0cbf8b93e741c2b500cbf1cb1ad5f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:23:34.139360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:23:34.139646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:23:34.139925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:23:34.148832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:23:34.150284 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:23:34.159610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1380/2730] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:23:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:23:44.051295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:23:44.155487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:02.156632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293d2f-3a7b-4a53-ac33-18cd719dceb4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:02.157140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293d2f-3a7b-4a53-ac33-18cd719dceb4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:02.157140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293d2f-3a7b-4a53-ac33-18cd719dceb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:02.176455 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d5293d2f-3a7b-4a53-ac33-18cd719dceb4 None None] MFA Rules not processed for user `f96ee29f9c5b4651825da16d0520a0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:02.177778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d5293d2f-3a7b-4a53-ac33-18cd719dceb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:02.242434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1351/2731] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:05.594767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:05.678153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6def61f-1e41-4ed4-ab96-02338e685346 service glance] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:05.683296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:05.684134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:05.684514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:05.684892 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] PATH_INFO: `/v3/users/bc9128f69a794f55b677500efbe87bf2` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:05.686592 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc9128f69a794f55b677500efbe87bf2)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:05.688388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:05.755474 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9128f69a794f55b677500efbe87bf2'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:05.770006 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9128f69a794f55b677500efbe87bf2'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:05.783251 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9128f69a794f55b677500efbe87bf2'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:05.791907 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9128f69a794f55b677500efbe87bf2'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:05.792632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9128f69a794f55b677500efbe87bf2'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:05.799019 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1381/2732] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:24:05 2022] DELETE /identity/v3/users/bc9128f69a794f55b677500efbe87bf2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:05.803596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:05.804114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:05.804209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:05.822500 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:05.825566 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:05.896969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1352/2733] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:24:05.907026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:05.908472 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:05.974717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d97acd-873c-49c2-ae55-cf413b711f62 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:05.978462 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d303c-2e6d-4d07-ba8f-e82d44a61cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:05.979240 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d303c-2e6d-4d07-ba8f-e82d44a61cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:05.979552 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d303c-2e6d-4d07-ba8f-e82d44a61cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:05.979831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d303c-2e6d-4d07-ba8f-e82d44a61cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:05.989532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d303c-2e6d-4d07-ba8f-e82d44a61cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:05.991031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d303c-2e6d-4d07-ba8f-e82d44a61cd0 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.000791 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1382/2734] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:06.270526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.272228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:06.288381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def62727-ba12-47f4-b829-2607a96fa04a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.294942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.295955 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.296305 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.296651 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] PATH_INFO: `/v3/projects/28bd315147274a3ca9174fc45a4fe861` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.298412 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bd315147274a3ca9174fc45a4fe861)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.300931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.357077 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bd315147274a3ca9174fc45a4fe861', 'request_id': 'req-e099ad46-40b4-41d3-a9fb-6ae99690eac2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:06.363124 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bd315147274a3ca9174fc45a4fe861', 'request_id': 'req-e099ad46-40b4-41d3-a9fb-6ae99690eac2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:06.369548 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bd315147274a3ca9174fc45a4fe861', 'request_id': 'req-e099ad46-40b4-41d3-a9fb-6ae99690eac2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:06.383232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] The token cache is being invalidate because project 28bd315147274a3ca9174fc45a4fe861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:24:06.383518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:06.384143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-e099ad46-40b4-41d3-a9fb-6ae99690eac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:06.385720 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1353/2735] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:24:06 2022] DELETE /identity/v3/projects/28bd315147274a3ca9174fc45a4fe861 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:06.391256 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.391655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.391815 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.410176 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:06.411404 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:06.430152 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.479183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1383/2736] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:06.487963 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.488259 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.488572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.511022 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:06.512684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:06.523907 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1354/2737] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:06.528498 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.540584 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ebbe0-947a-4e8f-9840-78cb4bc7c083 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.546258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.547596 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.547885 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.548162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.548921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:06.549245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.550901 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.563887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1384/2738] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:06.570184 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.580495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb20e8-a553-4842-aa0d-d4958ab702a5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.586446 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.587472 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.587747 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.588012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.588699 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.590609 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.614295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:06.622121 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1355/2739] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:24:06 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:06.628490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.642440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97994aba-d2ae-477e-afb7-dc4d06742a56 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.711685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.712797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.713143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.713477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.714400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.716611 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.739112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1385/2740] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:24:06 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:06.743191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.753708 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbedce8-6d8d-4ede-b71d-269b100a7c0f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.758641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.759364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.759650 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.759927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.760666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.762281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.773702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1356/2741] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:24:06.778060 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.788513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429770fb-d924-4924-9e44-8df7d5b1859b admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.793153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590413ca-0872-4069-ac88-fd992e06bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.793927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590413ca-0872-4069-ac88-fd992e06bb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.794213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590413ca-0872-4069-ac88-fd992e06bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.794487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590413ca-0872-4069-ac88-fd992e06bb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.795263 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590413ca-0872-4069-ac88-fd992e06bb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.797160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590413ca-0872-4069-ac88-fd992e06bb9b admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.809211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1386/2742] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:24:06.813429 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:06.833807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15e73d-a265-44f5-91cb-f43cb87252e0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:06.839575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:06.840490 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.840774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.841054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] PATH_INFO: `/v3/projects/18583b7aaaf54b32a789686de6d4c13b/users/32d78c76916f41dfa41631d122045fe3/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.856730 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=18583b7aaaf54b32a789686de6d4c13b, user_id=32d78c76916f41dfa41631d122045fe3, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:06.859523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:06.879551 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1357/2743] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:24:06 2022] PUT /identity/v3/projects/18583b7aaaf54b32a789686de6d4c13b/users/32d78c76916f41dfa41631d122045fe3/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:24:06.891244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:06.891244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:06.891575 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:06.923124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] MFA Rules not processed for user `32d78c76916f41dfa41631d122045fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:06.924488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:06.992918 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1387/2744] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:07.005173 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:07.007234 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:07.070376 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8507296-fe12-496c-8208-ef2d1fb18fdd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:07.074191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:07.074940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:07.075318 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:07.075695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:07.085241 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:07.087096 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:07.096774 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1358/2745] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:08.418076 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:08.419654 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:08.445580 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:08.500489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224722a-7518-44e0-b2ab-b1ae47a31095 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:08.504743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:08.505412 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:08.505715 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:08.505974 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:08.508335 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:08.582143 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:08.583653 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30b84b-abc5-4cf6-a215-ad284bea6564 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:08.593389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1388/2746] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:09.689218 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:09.778727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fb34b-5d8a-4e86-9290-0275209766c6 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:09.784020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:09.785015 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:09.785361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:09.785706 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] PATH_INFO: `/v3/users/32d78c76916f41dfa41631d122045fe3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:09.787134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d78c76916f41dfa41631d122045fe3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:09.789199 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:09.843079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d78c76916f41dfa41631d122045fe3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:09.860864 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d78c76916f41dfa41631d122045fe3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:09.866498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:09.875078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d78c76916f41dfa41631d122045fe3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:09.890159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d78c76916f41dfa41631d122045fe3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:09.891211 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d78c76916f41dfa41631d122045fe3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:09.898853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1359/2747] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:24:09 2022] DELETE /identity/v3/users/32d78c76916f41dfa41631d122045fe3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:09.903734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:09.904331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:09.904331 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:09.922431 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:09.924124 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:10.002133 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1389/2748] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:10.017636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.019697 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:10.080046 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201dbd-ef6a-4429-bf4f-6e1a81e69de3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.083855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1e52d-9811-4744-a092-dd556164d885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.084622 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1e52d-9811-4744-a092-dd556164d885 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.084948 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1e52d-9811-4744-a092-dd556164d885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.085289 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1e52d-9811-4744-a092-dd556164d885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.095836 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1e52d-9811-4744-a092-dd556164d885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.097386 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1e52d-9811-4744-a092-dd556164d885 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.107377 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1360/2749] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:10.419843 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.421320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:10.449388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ffe0c-1212-4c42-8510-f0a1ebd1aacf None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.455737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.456727 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.457134 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.457491 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] PATH_INFO: `/v3/projects/18583b7aaaf54b32a789686de6d4c13b` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.459160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18583b7aaaf54b32a789686de6d4c13b)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.462205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.529309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18583b7aaaf54b32a789686de6d4c13b', 'request_id': 'req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:10.537027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18583b7aaaf54b32a789686de6d4c13b', 'request_id': 'req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:10.549088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18583b7aaaf54b32a789686de6d4c13b', 'request_id': 'req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:10.554927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:10.562280 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] The token cache is being invalidate because project 18583b7aaaf54b32a789686de6d4c13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:24:10.563203 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:10.564310 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-12cc24c2-ad94-4b76-bc5f-a77af5e3f543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:10.566261 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1390/2750] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:24:10 2022] DELETE /identity/v3/projects/18583b7aaaf54b32a789686de6d4c13b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:10.572130 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.572445 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.572691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.595755 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:10.596994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:10.666209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1361/2751] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:10.674234 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.675490 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.676183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.727721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:10.729423 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:10.740055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1391/2752] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:10.744193 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.756118 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04672ed4-ed05-45d1-bd03-d7289213b66c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.761988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.762664 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.762992 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.763270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.764014 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:24:10.764322 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.766078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.779367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1362/2753] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:24:10.785229 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.796400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9857e-e46a-4653-8416-bdf0d4c45bfe None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.803121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.803808 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.804093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.804372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.805097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.807136 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.839416 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1392/2754] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:24:10 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:24:10.843470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.862299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9828a9-ca6d-47e8-9db2-42d5c3469e12 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:10.935810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:10.936565 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:10.936852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:10.937133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:10.938657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:10.941217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:10.979596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1363/2755] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:24:10 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:24:10.983867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:10.997493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e816e-929f-4bdb-82a5-9cc3cff20426 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:11.003308 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:11.004004 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:11.004295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:11.004574 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:11.005588 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:11.007248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:11.020935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1393/2756] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:24:11.025630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:11.038119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9755b-2108-4f91-abab-4796becc3249 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:11.043105 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d8919-a44b-42f7-bc9f-fb28fa2aa3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:11.043894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d8919-a44b-42f7-bc9f-fb28fa2aa3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:11.044180 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d8919-a44b-42f7-bc9f-fb28fa2aa3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:11.044462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d8919-a44b-42f7-bc9f-fb28fa2aa3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:11.046945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d8919-a44b-42f7-bc9f-fb28fa2aa3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:11.055477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d8919-a44b-42f7-bc9f-fb28fa2aa3b6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:11.057889 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:11.074159 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1364/2757] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:24:11.079091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:11.095390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c6db2-0e32-4b15-9f73-e4e7272d5923 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:11.101086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:11.102410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:11.102410 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:11.102597 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] PATH_INFO: `/v3/projects/5ea13d1f279d466e9a359c02548494b4/users/5ba18b0e8ba74364a104582d296334c6/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:11.120500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea13d1f279d466e9a359c02548494b4, user_id=5ba18b0e8ba74364a104582d296334c6, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:11.123354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:11.142756 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1394/2758] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:24:11 2022] PUT /identity/v3/projects/5ea13d1f279d466e9a359c02548494b4/users/5ba18b0e8ba74364a104582d296334c6/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:11.147596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33119c83-abc1-4e33-a8e5-e57e37fa32dc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:11.148434 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33119c83-abc1-4e33-a8e5-e57e37fa32dc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:11.148717 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33119c83-abc1-4e33-a8e5-e57e37fa32dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:11.167233 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-33119c83-abc1-4e33-a8e5-e57e37fa32dc None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:11.168841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-33119c83-abc1-4e33-a8e5-e57e37fa32dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:11.232387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1365/2759] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:11.242106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:11.243985 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:11.302312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fffb4d-d591-4af7-83bf-474446feeb8d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:11.306218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:11.306893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:11.307173 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:11.307440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:11.317827 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:11.319330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:11.329783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1395/2760] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:12.762395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:12.763045 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:12.763236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:12.795303 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] MFA Rules not processed for user `5ba18b0e8ba74364a104582d296334c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:12.796954 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:12.877908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1366/2761] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:12.889514 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:12.891776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:13.012549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1058a-f1bf-4726-9bb2-b9337a04fca3 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.017775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9280d24-43ea-47d0-a875-256e1931f4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.018464 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9280d24-43ea-47d0-a875-256e1931f4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.018784 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9280d24-43ea-47d0-a875-256e1931f4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.019064 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9280d24-43ea-47d0-a875-256e1931f4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.034371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9280d24-43ea-47d0-a875-256e1931f4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.036318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9280d24-43ea-47d0-a875-256e1931f4d2 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.050632 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1396/2762] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:13.632661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:13.642909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d5487-3e6e-4b30-8e6b-e9fdbd7277cc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:13.648052 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:13.648754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:13.650229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:13.650229 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:13.651188 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:13.664284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:13.665843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:13.676580 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1367/2763] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:14.487935 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:14.488537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:14.488537 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:14.507300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] MFA Rules not processed for user `5ba18b0e8ba74364a104582d296334c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:24:14.508638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:14.516829 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1397/2764] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:24:14.767901 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:14.769435 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:14.781099 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba2b9-3d1b-43a8-92aa-4d6e1a2b9aca service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:14.785940 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:14.786632 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:14.786994 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:14.787256 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:14.789224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:14.801139 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:14.802770 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:14.813152 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1368/2765] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:14.946165 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:14.947910 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:14.988619 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:15.030795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265a6be-726b-49e2-8cfd-02c0c603cdfe None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:15.036347 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:15.037069 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:15.037374 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:15.037673 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:15.039636 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:15.052327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:15.053936 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:15.064680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1398/2766] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:24:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:24:22.277965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:22.280003 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:22.297898 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c437b-9367-426a-986b-9187e109a887 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:22.302956 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:22.303841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:22.304122 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:22.304391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] PATH_INFO: `/v3/users/23c7ecdb284d4ad0bda072a87ad8984c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:22.305600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c7ecdb284d4ad0bda072a87ad8984c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:22.307602 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:22.358249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c7ecdb284d4ad0bda072a87ad8984c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.363815 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:22.371991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c7ecdb284d4ad0bda072a87ad8984c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.379712 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c7ecdb284d4ad0bda072a87ad8984c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.387092 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c7ecdb284d4ad0bda072a87ad8984c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.387760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c7ecdb284d4ad0bda072a87ad8984c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.394476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1369/2767] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:24:22 2022] DELETE /identity/v3/users/23c7ecdb284d4ad0bda072a87ad8984c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:22.593638 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:22.606005 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e7b52-5a35-4095-9069-970cccfb210b service placement] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:22.698318 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:22.699408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:22.699820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:22.700192 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] PATH_INFO: `/v3/projects/90f6878fdaaf43b58b92bea357331d2e` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:22.701520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f6878fdaaf43b58b92bea357331d2e)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:22.703775 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:22.757187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f6878fdaaf43b58b92bea357331d2e', 'request_id': 'req-823e28fd-3c74-4aa1-93cd-98230c2c94a4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.763643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f6878fdaaf43b58b92bea357331d2e', 'request_id': 'req-823e28fd-3c74-4aa1-93cd-98230c2c94a4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.772590 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f6878fdaaf43b58b92bea357331d2e', 'request_id': 'req-823e28fd-3c74-4aa1-93cd-98230c2c94a4'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.783955 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] The token cache is being invalidate because project 90f6878fdaaf43b58b92bea357331d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:24:22.784238 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.784867 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.786591 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1399/2768] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:24:22 2022] DELETE /identity/v3/projects/90f6878fdaaf43b58b92bea357331d2e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:22.790665 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:22.792478 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:22.806754 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996aa597-3ddf-41c8-8520-e5f892a03673 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:22.866659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:22.867572 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:22.867862 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:22.868136 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] PATH_INFO: `/v3/users/72de2072aa2e432380334a37e9061231` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:22.869345 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72de2072aa2e432380334a37e9061231)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:22.871349 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:22.925450 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72de2072aa2e432380334a37e9061231'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.940153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72de2072aa2e432380334a37e9061231'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.949746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72de2072aa2e432380334a37e9061231'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.961285 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72de2072aa2e432380334a37e9061231'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.962334 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0e28b304-9825-4115-bf89-f4f9233dda01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72de2072aa2e432380334a37e9061231'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:22.969417 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1370/2769] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:24:22 2022] DELETE /identity/v3/users/72de2072aa2e432380334a37e9061231 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:23.165296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:24:23.166870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:24:23.186603 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e28fd-3c74-4aa1-93cd-98230c2c94a4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:24:23.243419 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:23.256182 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:24:23.257434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:24:23.257975 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:24:23.258595 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] PATH_INFO: `/v3/projects/ec8925e14c8245d8a8ea20df16b39f36` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:24:23.259824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8925e14c8245d8a8ea20df16b39f36)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:24:23.261764 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:24:23.312411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8925e14c8245d8a8ea20df16b39f36', 'request_id': 'req-696ddfd4-539f-4971-b505-63d1ed82cb61'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:23.319216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8925e14c8245d8a8ea20df16b39f36', 'request_id': 'req-696ddfd4-539f-4971-b505-63d1ed82cb61'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:23.327840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8925e14c8245d8a8ea20df16b39f36', 'request_id': 'req-696ddfd4-539f-4971-b505-63d1ed82cb61'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:23.339040 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] The token cache is being invalidate because project ec8925e14c8245d8a8ea20df16b39f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:24:23.339294 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:23.339909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-696ddfd4-539f-4971-b505-63d1ed82cb61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:24:23.341496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1400/2770] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:24:23 2022] DELETE /identity/v3/projects/ec8925e14c8245d8a8ea20df16b39f36 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:24:32.965057 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=63,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:24:33.335649 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:07.793527 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:07.794061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:07.794061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:07.812905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] MFA Rules not processed for user `f96ee29f9c5b4651825da16d0520a0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:25:07.814153 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:25:07.873134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1371/2771] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:25:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:25:15.002200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:25:15.002806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:25:15.002806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:25:15.021281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] MFA Rules not processed for user `5ba18b0e8ba74364a104582d296334c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:25:15.022496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:25:15.083894 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1401/2772] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:25:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:25:17.865850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=25 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:25:25.076428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=25 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:10.044991 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:10.046509 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:10.120825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf218-1cf4-4357-9650-8f131df3964c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:10.125592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:10.126463 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.126769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.127031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] PATH_INFO: `/v3/users/5ba18b0e8ba74364a104582d296334c6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.128372 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba18b0e8ba74364a104582d296334c6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:10.130283 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:10.181896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba18b0e8ba74364a104582d296334c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.194931 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba18b0e8ba74364a104582d296334c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.203268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba18b0e8ba74364a104582d296334c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.211321 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba18b0e8ba74364a104582d296334c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.212004 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fbe3d8d9-7816-47de-9c90-591369aac427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba18b0e8ba74364a104582d296334c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.218820 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1372/2773] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:26:10 2022] DELETE /identity/v3/users/5ba18b0e8ba74364a104582d296334c6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:10.652123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:10.663453 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c327d3-beb2-4d7f-890a-61d90bcf4aa5 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:10.730573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:10.731489 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.731767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.732034 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] PATH_INFO: `/v3/projects/5ea13d1f279d466e9a359c02548494b4` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.733281 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea13d1f279d466e9a359c02548494b4)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:10.735271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:10.790296 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea13d1f279d466e9a359c02548494b4', 'request_id': 'req-d7209403-9a9b-4f82-b4c1-f995243c3f33'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.797943 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea13d1f279d466e9a359c02548494b4', 'request_id': 'req-d7209403-9a9b-4f82-b4c1-f995243c3f33'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.808326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea13d1f279d466e9a359c02548494b4', 'request_id': 'req-d7209403-9a9b-4f82-b4c1-f995243c3f33'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.819832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] The token cache is being invalidate because project 5ea13d1f279d466e9a359c02548494b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80295) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:26:10.820200 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.820898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d7209403-9a9b-4f82-b4c1-f995243c3f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:10.822470 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1402/2774] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:26:10 2022] DELETE /identity/v3/projects/5ea13d1f279d466e9a359c02548494b4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:10.828299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.828586 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.828982 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.848115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:10.849279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:10.910306 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1373/2775] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:10.918548 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.918949 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.919290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.941919 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:10.943438 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:10.952385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1403/2776] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:10.956824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:10.967905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d4d35-ccc9-4e8d-8af2-123039e18c4f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:10.973399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:10.974068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:10.974342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:10.974764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:10.975378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:10.975684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:10.977364 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:10.993078 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1374/2777] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:10.998166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.013970 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaebac3-040b-46b9-9c15-fa1c37a847aa None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.019675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.020434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.020768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.021095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.022475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.024662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.062998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1404/2778] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:26:10 2022] POST /identity/v3/projects => generated 395 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:11.064367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.079163 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3495d85-8d00-4a9b-8f72-9b445e2ba119 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.084518 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.154896 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.156227 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.156599 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.156983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.157912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.160002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.193563 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.194909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1375/2779] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:26:11 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:11.216895 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66b57-c1aa-4047-be3b-0c39ad23e619 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.222454 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.223461 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.223937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.224276 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.225086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.226881 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.246965 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1405/2780] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:11.248353 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.260143 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81659-3710-48dd-9852-536e292fcd2f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.265544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709aa19c-fdf1-455a-b1c9-5290749e7a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.266404 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aa19c-fdf1-455a-b1c9-5290749e7a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.267614 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aa19c-fdf1-455a-b1c9-5290749e7a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.268001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709aa19c-fdf1-455a-b1c9-5290749e7a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.268947 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709aa19c-fdf1-455a-b1c9-5290749e7a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.271314 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709aa19c-fdf1-455a-b1c9-5290749e7a2d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.284736 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1376/2781] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:11.288916 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.301183 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdafc06-3748-4457-86cc-bcf815f151fe admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.307088 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.308969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] REQUEST_METHOD: `PUT` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.309247 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.309519 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] PATH_INFO: `/v3/projects/7c63b04a1a014a829841614d1d9db482/users/b42fcb48958a430bb5ee0f32c853ae3c/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.327582 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c63b04a1a014a829841614d1d9db482, user_id=b42fcb48958a430bb5ee0f32c853ae3c, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.330458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.347422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1406/2782] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:26:11 2022] PUT /identity/v3/projects/7c63b04a1a014a829841614d1d9db482/users/b42fcb48958a430bb5ee0f32c853ae3c/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:11.358823 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3b4-e1fa-4892-a7af-54369b3f28f0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.359116 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3b4-e1fa-4892-a7af-54369b3f28f0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.359395 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3b4-e1fa-4892-a7af-54369b3f28f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.386111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-988dc3b4-e1fa-4892-a7af-54369b3f28f0 None None] MFA Rules not processed for user `b42fcb48958a430bb5ee0f32c853ae3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:11.387397 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-988dc3b4-e1fa-4892-a7af-54369b3f28f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:11.455936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1377/2783] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:11.467806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:11.469698 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:11.508663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:11.544703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b62a34-75fe-4e44-9c47-2e93841d37be admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:11.551577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:11.552446 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:11.553546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:11.553546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:11.565071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:11.566131 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:11.577357 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1407/2784] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:26:14.169451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b7f5-a6c4-474b-8c86-c72ea04fe13f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:14.170040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b7f5-a6c4-474b-8c86-c72ea04fe13f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:14.170040 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b7f5-a6c4-474b-8c86-c72ea04fe13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:14.188451 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9cb0b7f5-a6c4-474b-8c86-c72ea04fe13f None None] MFA Rules not processed for user `f96ee29f9c5b4651825da16d0520a0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:14.189666 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9cb0b7f5-a6c4-474b-8c86-c72ea04fe13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:14.249719 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1378/2785] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:18.667724 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:18.737435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428ffc8-ba26-4037-b3ec-99e8ffb1ed49 service cinder] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:18.742414 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:18.743480 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:18.743759 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:18.744025 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] PATH_INFO: `/v3/users/b42fcb48958a430bb5ee0f32c853ae3c` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:18.745583 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42fcb48958a430bb5ee0f32c853ae3c)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:18.747570 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:18.795925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42fcb48958a430bb5ee0f32c853ae3c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:18.810921 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42fcb48958a430bb5ee0f32c853ae3c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:18.819550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42fcb48958a430bb5ee0f32c853ae3c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:18.826723 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42fcb48958a430bb5ee0f32c853ae3c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:18.827360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42fcb48958a430bb5ee0f32c853ae3c'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:18.833663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1408/2786] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:26:18 2022] DELETE /identity/v3/users/b42fcb48958a430bb5ee0f32c853ae3c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:18.838590 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:18.839852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:18.839852 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:18.857568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:18.858767 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:18.872976 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:18.920657 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1379/2787] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:26:18.930617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:18.932115 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:18.988578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:18.992760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15509d8-10d2-4502-a00c-2c21efe0e2c9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:18.997364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716288c-ed41-45d1-aeaa-d74bae00fc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:18.998031 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716288c-ed41-45d1-aeaa-d74bae00fc9e service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:18.998319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716288c-ed41-45d1-aeaa-d74bae00fc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:18.998600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716288c-ed41-45d1-aeaa-d74bae00fc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.008797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716288c-ed41-45d1-aeaa-d74bae00fc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.010244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716288c-ed41-45d1-aeaa-d74bae00fc9e service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.020501 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1409/2788] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:26:19.415875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.418056 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:19.431091 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d610380-46fd-4448-87c3-afc7f2ffaf50 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.437002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.437930 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.438209 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.438476 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] PATH_INFO: `/v3/projects/7c63b04a1a014a829841614d1d9db482` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.439833 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c63b04a1a014a829841614d1d9db482)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.441769 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.512275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c63b04a1a014a829841614d1d9db482', 'request_id': 'req-989bbc04-458d-4797-a30d-328f18b4a922'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:19.518461 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c63b04a1a014a829841614d1d9db482', 'request_id': 'req-989bbc04-458d-4797-a30d-328f18b4a922'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:19.525631 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c63b04a1a014a829841614d1d9db482', 'request_id': 'req-989bbc04-458d-4797-a30d-328f18b4a922'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:19.540001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] The token cache is being invalidate because project 7c63b04a1a014a829841614d1d9db482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:26:19.540348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:19.541147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-989bbc04-458d-4797-a30d-328f18b4a922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:26:19.543013 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1380/2789] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:26:19 2022] DELETE /identity/v3/projects/7c63b04a1a014a829841614d1d9db482 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:19.548383 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.548998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.548998 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.568339 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:19.569576 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:19.634641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1410/2790] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:19.643235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.644249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.644600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.666087 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:19.667850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:19.677272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1381/2791] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:19.681903 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.693573 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786e38a-03a6-46cb-9b3a-ebffbc3bfa6c None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.699132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.699821 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.700100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.700364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.701107 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:26:19.701415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.703091 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.721648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1411/2792] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:26:19.726297 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.737051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0878e-edb0-4dd9-9077-5065aafc62dd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.742281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.743028 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.743308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.743591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.744911 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.746541 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.775922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1382/2793] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:26:19 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:19.777509 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.788077 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37cc59-7d80-4045-b138-22c31be3d7f7 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.846073 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.846945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.847363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.847717 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.848521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.850507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.874982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1412/2794] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:26:19 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:19.876751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.888608 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bc7e6-7b32-4a0f-b378-b04702d29a12 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.893738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.894422 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.894735 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.895162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.895942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.897751 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.909648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1383/2795] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:19.913883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.925168 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d6ac8-f537-40c1-ac6a-70a52fcce3a4 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.930424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdd25c-d2cd-4b10-a979-c487ccb0eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.931122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdd25c-d2cd-4b10-a979-c487ccb0eff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.931433 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdd25c-d2cd-4b10-a979-c487ccb0eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.931706 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdd25c-d2cd-4b10-a979-c487ccb0eff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.932623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdd25c-d2cd-4b10-a979-c487ccb0eff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.934214 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdd25c-d2cd-4b10-a979-c487ccb0eff3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:19.944637 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1413/2796] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:19.948980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:19.959385 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e8209-7e7d-4ae0-a96f-3ddbf3373ac1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:19.964571 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:19.965471 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:19.965758 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:19.966032 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] PATH_INFO: `/v3/projects/a1af4242504b4c7fa0903c7c61b02135/users/5855c9996e4345aa8f71463648b6009e/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:19.982691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1af4242504b4c7fa0903c7c61b02135, user_id=5855c9996e4345aa8f71463648b6009e, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:19.985498 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:20.001157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1384/2797] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:26:19 2022] PUT /identity/v3/projects/a1af4242504b4c7fa0903c7c61b02135/users/5855c9996e4345aa8f71463648b6009e/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:20.006265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3ae33-8744-4274-945e-78297437bb4c None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:20.006561 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3ae33-8744-4274-945e-78297437bb4c None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:20.006876 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3ae33-8744-4274-945e-78297437bb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:20.025722 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f0a3ae33-8744-4274-945e-78297437bb4c None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:20.027486 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f0a3ae33-8744-4274-945e-78297437bb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:20.080617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:20.091757 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1414/2798] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:20.101642 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:20.103535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:20.150253 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:20.166365 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e936e1e-5b77-4fc3-b99a-094d7599c053 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:20.170298 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:20.171340 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:20.171536 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:20.171811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:20.181800 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:20.183578 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:20.194029 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1385/2799] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:26:24.425299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.425887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.425887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.430593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.432465 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:24.445634 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418486cb-56ee-4eed-9c99-820c2fcc9126 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.450824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.451592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.451945 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.452220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.454021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:24.460972 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] MFA Rules not processed for user `5855c9996e4345aa8f71463648b6009e`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:24.462305 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:24.548248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1415/2800] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:24.552962 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.554171 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.556540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.566012 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20a972-5777-40ed-a893-48940bd6a691 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.570760 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1386/2801] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:26:24.574992 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.575830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.576216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.576565 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] PATH_INFO: `/v3/projects` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.577364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.579482 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.611156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1416/2802] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:26:24 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:26:24.614846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.626427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefe95-4d4d-4e65-8253-c4d1c226fdb3 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.702174 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.703007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.703350 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.707690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.707690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.707690 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.733829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1387/2803] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:26:24 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:26:24.737290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.748840 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f75ba-2be4-4e88-8185-777953c1e542 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.753630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.754320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.754600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.754928 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.755672 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.757390 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.774439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1417/2804] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:24.775860 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.786875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdf6e5-f870-4d4c-a644-e676824c501e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.792021 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.793072 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.793358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.793663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] PATH_INFO: `/v3/projects/de9aedf5d1a74ed88a6e2c013e13e849/users/ece2d66fe73c424fbd8947ea8e927957/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.810801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9aedf5d1a74ed88a6e2c013e13e849, user_id=ece2d66fe73c424fbd8947ea8e927957, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.813575 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.839776 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.841192 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1388/2805] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:26:24 2022] PUT /identity/v3/projects/de9aedf5d1a74ed88a6e2c013e13e849/users/ece2d66fe73c424fbd8947ea8e927957/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:24.855277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bf3bc-0c9f-45f7-a0e7-c0b1c726dc30 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.927079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02e88b-9198-4b12-822a-9bb815a27f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.927770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02e88b-9198-4b12-822a-9bb815a27f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.928054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02e88b-9198-4b12-822a-9bb815a27f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.928330 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02e88b-9198-4b12-822a-9bb815a27f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.929070 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02e88b-9198-4b12-822a-9bb815a27f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.930719 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02e88b-9198-4b12-822a-9bb815a27f59 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.941539 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1418/2806] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:26:24.945726 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:24.956646 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960a286-5a61-47c8-b3cf-14fc6c27d6e4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:24.961848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:24.962857 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:24.963231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:24.963641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] PATH_INFO: `/v3/projects/de9aedf5d1a74ed88a6e2c013e13e849/users/ece2d66fe73c424fbd8947ea8e927957/roles/b10ab4271e1845e4aefab70d82a12def` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:24.966454 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9aedf5d1a74ed88a6e2c013e13e849, user_id=ece2d66fe73c424fbd8947ea8e927957, role_id=b10ab4271e1845e4aefab70d82a12def)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:24.969401 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:24.980546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1389/2807] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:26:24 2022] PUT /identity/v3/projects/de9aedf5d1a74ed88a6e2c013e13e849/users/ece2d66fe73c424fbd8947ea8e927957/roles/b10ab4271e1845e4aefab70d82a12def => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:26:29.870327 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7f064-6ba0-477d-a1b4-f69602862a09 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:29.871290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7f064-6ba0-477d-a1b4-f69602862a09 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:29.871872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7f064-6ba0-477d-a1b4-f69602862a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:29.907857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-23c7f064-6ba0-477d-a1b4-f69602862a09 None None] MFA Rules not processed for user `ece2d66fe73c424fbd8947ea8e927957`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:29.909104 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-23c7f064-6ba0-477d-a1b4-f69602862a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:29.966377 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:30.013066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1419/2808] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:30.028810 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:30.030802 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:30.056067 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:30.115368 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e655ed6-642f-4478-be01-3a6ccffeae0b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:30.121114 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:30.121775 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:30.122049 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:30.122319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:30.183308 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:30.184965 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:30.195115 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1390/2809] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:26:40.009703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=11 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:40.190458 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=47 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:26:46.603758 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:46.604338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:46.604338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:46.624016 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] MFA Rules not processed for user `5855c9996e4345aa8f71463648b6009e`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:26:46.625411 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:46.633479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1420/2810] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:26:47.481628 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:47.541018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1832e0-13de-4962-be8b-d4740884142e service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:47.544874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:47.545491 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:47.545793 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:47.546080 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:47.548423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:47.563828 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:47.565342 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:47.577304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1391/2811] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:26:50.055798 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:50.057957 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:50.080792 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b283b-b7af-4560-9e99-444442fc1cf2 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:50.088039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:50.088948 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:50.089338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:50.089819 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:50.092549 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:50.108816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:50.110902 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:50.120660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:26:50.124211 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1421/2812] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:26:50.135236 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670c0c8-96d0-4248-a98d-37cd07bbd78d service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:26:50.139475 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:26:50.140157 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:26:50.140441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:26:50.140721 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:26:50.142804 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:26:50.154291 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:26:50.155638 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:26:50.169178 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1392/2813] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:27:00.118809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=9 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:00.162017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=32 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:03.115520 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:03.125814 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec7b8f-846e-4639-a321-a4ac0ed5dbb8 service nova] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:03.143564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:03.144257 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:03.144541 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:03.144816 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:03.158002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:03.159924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:03.171112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1422/2814] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:05.086354 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:05.101270 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3a84-9658-4519-9cde-4c1969465843 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:05.106983 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:05.107641 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:05.107916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:05.108201 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:05.109999 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:05.122871 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:05.124473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:05.135845 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1393/2815] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:13.166054 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=6 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:15.131197 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:20.394659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:20.396260 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:20.466283 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f91280-edf2-4e95-96bc-aba28e6b6cd5 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:20.471312 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:20.472204 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:20.472488 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:20.472768 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] PATH_INFO: `/v3/users/f96ee29f9c5b4651825da16d0520a0e8` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:20.474018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96ee29f9c5b4651825da16d0520a0e8)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:20.476001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:20.524400 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96ee29f9c5b4651825da16d0520a0e8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.537477 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96ee29f9c5b4651825da16d0520a0e8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.546248 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96ee29f9c5b4651825da16d0520a0e8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.554422 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96ee29f9c5b4651825da16d0520a0e8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.555585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-7b1350db-8b85-4f4e-a6c2-2e03839671eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96ee29f9c5b4651825da16d0520a0e8'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.562066 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1423/2816] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:27:20 2022] DELETE /identity/v3/users/f96ee29f9c5b4651825da16d0520a0e8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:20.749035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:20.761277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f803-225a-4470-92c2-3542daebe2b4 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:20.821181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:20.821964 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:20.822249 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:20.822524 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] PATH_INFO: `/v3/projects/13b216e0dd4e46e2a2f4b275666118f6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:20.823757 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b216e0dd4e46e2a2f4b275666118f6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:20.825698 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:20.875486 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b216e0dd4e46e2a2f4b275666118f6', 'request_id': 'req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.881374 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b216e0dd4e46e2a2f4b275666118f6', 'request_id': 'req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.887925 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b216e0dd4e46e2a2f4b275666118f6', 'request_id': 'req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.903683 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] The token cache is being invalidate because project 13b216e0dd4e46e2a2f4b275666118f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:27:20.903975 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.904655 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3af9346c-e1f5-4ffc-b6ac-6f6d41f29782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:20.906304 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1394/2817] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:27:20 2022] DELETE /identity/v3/projects/13b216e0dd4e46e2a2f4b275666118f6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:20.911652 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:20.911946 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:20.912225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:20.931259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:20.932458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:20.995877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1424/2818] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:21.004089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.004394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.004684 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.024573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:21.025850 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:21.035649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1395/2819] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:21.040129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.050656 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33886942-6a59-43cc-a37b-0312f64bea31 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.055721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.056393 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.056644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.056924 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.057663 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:27:21.057973 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.059807 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.072550 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1425/2820] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:27:21.076997 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.087194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8f05d-082a-4a30-8331-687d267d9d85 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.092564 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.093278 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.093562 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.093837 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.094546 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.096849 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.130113 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1396/2821] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:27:21 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:21.134681 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.145542 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b711-f73e-46f0-89d8-2a5a2988daaf admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.149529 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:21.209041 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.209774 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.210055 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.210326 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.211039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.213030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.237049 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1426/2822] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:27:21 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:21.241094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.252190 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeaa70a-3978-4cf2-aeb3-dcf474c47db5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.258739 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.259466 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.259744 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.260018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.260763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.262403 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.274821 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1397/2823] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:21.279213 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.291154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa378-e66a-41b3-bb11-7916b50a93af admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.296315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857a762-ef2b-48a7-953a-a74cfb0fa2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.296993 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857a762-ef2b-48a7-953a-a74cfb0fa2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.297275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857a762-ef2b-48a7-953a-a74cfb0fa2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.297556 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857a762-ef2b-48a7-953a-a74cfb0fa2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.298443 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857a762-ef2b-48a7-953a-a74cfb0fa2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.300309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857a762-ef2b-48a7-953a-a74cfb0fa2d0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.310889 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1427/2824] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:21.315333 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.325592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2293d02-22ab-41bc-be6d-7cc937dd1d51 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.330582 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.331593 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.331875 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.332338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] PATH_INFO: `/v3/projects/f4fed92cd7454c0b8c682408358d1edc/users/33dc946289b94e57b10472973bcdf3cf/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.349406 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fed92cd7454c0b8c682408358d1edc, user_id=33dc946289b94e57b10472973bcdf3cf, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.352165 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.370254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1398/2825] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:27:21 2022] PUT /identity/v3/projects/f4fed92cd7454c0b8c682408358d1edc/users/33dc946289b94e57b10472973bcdf3cf/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:21.380643 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.380945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.381230 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.408900 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] MFA Rules not processed for user `33dc946289b94e57b10472973bcdf3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:21.410160 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:21.477129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1428/2826] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:21.490544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:21.492464 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:21.523535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:21.569788 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737988f-35a4-486b-ab11-319943e1e3b3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:21.576856 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0aba-4340-4f99-861d-3374af02eb01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:21.577470 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0aba-4340-4f99-861d-3374af02eb01 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:21.577752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0aba-4340-4f99-861d-3374af02eb01 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:21.578024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0aba-4340-4f99-861d-3374af02eb01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:21.587394 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0aba-4340-4f99-861d-3374af02eb01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:21.589007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0aba-4340-4f99-861d-3374af02eb01 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:21.598473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1399/2827] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:24.292512 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:24.358578 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568702c-03ee-4474-81f7-c917428d3365 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:24.363826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:24.364714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:24.364979 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:24.365279 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] PATH_INFO: `/v3/users/33dc946289b94e57b10472973bcdf3cf` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:24.366521 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33dc946289b94e57b10472973bcdf3cf)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:24.368528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:24.387909 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:24.421354 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33dc946289b94e57b10472973bcdf3cf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:24.433710 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33dc946289b94e57b10472973bcdf3cf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:24.442249 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33dc946289b94e57b10472973bcdf3cf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:24.449627 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33dc946289b94e57b10472973bcdf3cf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:24.450360 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33dc946289b94e57b10472973bcdf3cf'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:24.457988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1429/2828] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:27:24 2022] DELETE /identity/v3/users/33dc946289b94e57b10472973bcdf3cf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:24.463083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:24.463606 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:24.463727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:24.482269 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:24.483507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:24.546103 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1400/2829] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:24.556720 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:24.558425 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:24.618883 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354c7cc-2614-420b-973a-849387f35aa0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:24.623594 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc95ca7-da91-4590-8b37-e1baf952a6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:24.624697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc95ca7-da91-4590-8b37-e1baf952a6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:24.624697 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc95ca7-da91-4590-8b37-e1baf952a6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:24.625074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc95ca7-da91-4590-8b37-e1baf952a6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:24.635795 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc95ca7-da91-4590-8b37-e1baf952a6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:24.637273 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc95ca7-da91-4590-8b37-e1baf952a6e6 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:24.647079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1430/2830] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:27:24.931532 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:24.933005 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:24.946296 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b592e-6c74-4c87-9d74-27d54dbbe1ac None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:24.953268 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:24.954148 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:24.954430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:24.954872 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] PATH_INFO: `/v3/projects/f4fed92cd7454c0b8c682408358d1edc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:24.956559 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fed92cd7454c0b8c682408358d1edc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:24.958516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.010282 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fed92cd7454c0b8c682408358d1edc', 'request_id': 'req-1c343700-8e92-4549-8754-87a4a467f9b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:25.015894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fed92cd7454c0b8c682408358d1edc', 'request_id': 'req-1c343700-8e92-4549-8754-87a4a467f9b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:25.023626 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fed92cd7454c0b8c682408358d1edc', 'request_id': 'req-1c343700-8e92-4549-8754-87a4a467f9b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:25.038908 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] The token cache is being invalidate because project f4fed92cd7454c0b8c682408358d1edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:27:25.039194 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:25.039855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c343700-8e92-4549-8754-87a4a467f9b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:25.041438 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1401/2831] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:27:24 2022] DELETE /identity/v3/projects/f4fed92cd7454c0b8c682408358d1edc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:25.046832 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.047129 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.047436 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.066648 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:25.067826 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:25.133497 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1431/2832] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:25.142933 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.143232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.143516 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.163789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:25.165062 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:25.173885 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1402/2833] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:25.178660 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.189071 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c0124-f233-46ff-8299-7cc1ff7892e4 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.194676 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.195372 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.195655 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.195927 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.196661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:27:25.196967 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.198610 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.211355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1432/2834] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:27:25.215916 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.225603 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aeff7-5d82-4bde-beec-40780c1c8fa8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.230950 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.231596 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.231848 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.232119 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.232819 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.234825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.262980 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1403/2835] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:27:25 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:27:25.267338 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.277831 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8573a-99f9-48a9-b9cd-6e8cc4e8d2fb admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.299897 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:25.340271 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.340866 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.341153 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.341434 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.342132 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.344106 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.366212 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1433/2836] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:27:25 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:25.370468 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.381358 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e755edb-bf85-4760-96f7-835feeb417d5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.385987 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.386644 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.386957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.387238 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.387973 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.389576 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.390894 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:25.402789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1404/2837] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:25.407151 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.418171 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7913-b66b-4b37-9db7-dd4b8c9d6942 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.424046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf00249-cedd-454f-ab07-3ce87a1e4176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.424734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00249-cedd-454f-ab07-3ce87a1e4176 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.425018 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00249-cedd-454f-ab07-3ce87a1e4176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.425295 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf00249-cedd-454f-ab07-3ce87a1e4176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.426033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf00249-cedd-454f-ab07-3ce87a1e4176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.427692 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf00249-cedd-454f-ab07-3ce87a1e4176 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.437734 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1434/2838] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:25.442378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.453017 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc266a1-6c0b-4b57-8fa2-7738089be6b9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.459381 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.460272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.460553 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.460829 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] PATH_INFO: `/v3/projects/411d7bb7706b460593e96a49cbd4b0df/users/64696d7676744506b938a2f64c1fefca/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.476784 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=411d7bb7706b460593e96a49cbd4b0df, user_id=64696d7676744506b938a2f64c1fefca, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.479515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.493765 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1405/2839] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:27:25 2022] PUT /identity/v3/projects/411d7bb7706b460593e96a49cbd4b0df/users/64696d7676744506b938a2f64c1fefca/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:25.504301 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.504585 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.504893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.533388 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] MFA Rules not processed for user `64696d7676744506b938a2f64c1fefca`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:25.534615 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:25.604126 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1435/2840] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:25.616012 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:25.617874 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:25.693870 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18963e-b629-4998-9602-4b2cbf1b15c1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:25.700495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f2c7-0ab2-4796-b158-7bb794191c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:25.701149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f2c7-0ab2-4796-b158-7bb794191c69 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:25.701428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f2c7-0ab2-4796-b158-7bb794191c69 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:25.701694 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f2c7-0ab2-4796-b158-7bb794191c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:25.711483 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f2c7-0ab2-4796-b158-7bb794191c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:25.713089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f2c7-0ab2-4796-b158-7bb794191c69 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:25.723061 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1406/2841] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:25.955839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:26.024939 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe86e3-eca6-49e6-8282-2b9cb7988729 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:26.030380 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:26.031320 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.031606 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.031884 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] PATH_INFO: `/v3/users/64696d7676744506b938a2f64c1fefca` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.033172 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64696d7676744506b938a2f64c1fefca)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:26.035177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:26.076262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:26.088188 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64696d7676744506b938a2f64c1fefca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.100703 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64696d7676744506b938a2f64c1fefca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.109783 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64696d7676744506b938a2f64c1fefca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.118824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64696d7676744506b938a2f64c1fefca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.119600 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64696d7676744506b938a2f64c1fefca'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.126278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1436/2842] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:27:25 2022] DELETE /identity/v3/users/64696d7676744506b938a2f64c1fefca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:26.131620 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.131904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.132362 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.156020 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:26.157275 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:26.221373 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1407/2843] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:26.231418 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:26.233303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:26.296440 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfa4f2-dc00-4e8a-a1e0-3b458d3f2790 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:26.300952 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d9c11-9ab3-4c96-a830-6fe0b2fb1375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:26.301564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d9c11-9ab3-4c96-a830-6fe0b2fb1375 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.301879 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d9c11-9ab3-4c96-a830-6fe0b2fb1375 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.302154 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d9c11-9ab3-4c96-a830-6fe0b2fb1375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.312157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d9c11-9ab3-4c96-a830-6fe0b2fb1375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:26.313604 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d9c11-9ab3-4c96-a830-6fe0b2fb1375 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:26.324342 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1437/2844] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:26.614503 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:26.616031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:26.633654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0008d-9474-44e5-aa53-e722bb975cd4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:26.639361 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:26.640224 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.640507 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.640853 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] PATH_INFO: `/v3/projects/411d7bb7706b460593e96a49cbd4b0df` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.642247 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=411d7bb7706b460593e96a49cbd4b0df)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:26.644416 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:26.694391 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411d7bb7706b460593e96a49cbd4b0df', 'request_id': 'req-300807f8-fd36-49e9-a284-7ffd8f8596bc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.698264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:26.700648 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411d7bb7706b460593e96a49cbd4b0df', 'request_id': 'req-300807f8-fd36-49e9-a284-7ffd8f8596bc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.707768 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411d7bb7706b460593e96a49cbd4b0df', 'request_id': 'req-300807f8-fd36-49e9-a284-7ffd8f8596bc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.722195 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] The token cache is being invalidate because project 411d7bb7706b460593e96a49cbd4b0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:27:26.722502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.723162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-300807f8-fd36-49e9-a284-7ffd8f8596bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:26.724738 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1408/2845] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:27:26 2022] DELETE /identity/v3/projects/411d7bb7706b460593e96a49cbd4b0df => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:27:26.730969 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.731355 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.731641 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.750021 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:26.751319 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:26.813221 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1438/2846] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:26.821789 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.822079 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.822359 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.841927 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:26.843214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:26.851941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1409/2847] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:26.856593 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:26.866991 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473cfce-762f-4cd1-92bb-e9adf49d46e3 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:26.872670 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:26.873336 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.873622 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.873904 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.874639 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:27:26.874986 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:26.876914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:26.889513 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1439/2848] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:27:26.893936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:26.903669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963d315-5815-41fe-829d-82820dde7d54 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:26.908827 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:26.909496 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:26.909776 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:26.910054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:26.910787 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:26.912709 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:26.940830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1410/2849] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:27:26 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:26.945225 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:26.956452 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572b613-ac57-4f90-9f14-1f582d523a9d admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:26.999780 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:27.020162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.020845 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.021123 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.021394 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.022092 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.024138 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.047082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1440/2850] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:27:26 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:27.051543 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:27.062635 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07159-241a-4a2a-8a83-4f11015a1d05 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:27.068942 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.069643 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.069936 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.070203 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.071007 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.072663 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.084573 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1411/2851] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:27.089206 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:27.100074 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aca9a-dbe6-4ae5-94cd-745bbf774ad0 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:27.105142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66f1e2-d361-4305-bea6-1f83ddc7de79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.105810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f1e2-d361-4305-bea6-1f83ddc7de79 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.106095 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f1e2-d361-4305-bea6-1f83ddc7de79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.106367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f1e2-d361-4305-bea6-1f83ddc7de79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.107139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f1e2-d361-4305-bea6-1f83ddc7de79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.108748 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f1e2-d361-4305-bea6-1f83ddc7de79 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.122102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1441/2852] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:27.126660 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:27.139287 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea36b-d67d-48a7-b81b-fb7247603848 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:27.145018 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.145947 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.146230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.146502 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] PATH_INFO: `/v3/projects/0c212876ec464b56a48d068544e1c8c7/users/3428ab860bff4695804b00f2eb35d4ae/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.163068 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c212876ec464b56a48d068544e1c8c7, user_id=3428ab860bff4695804b00f2eb35d4ae, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.165811 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.180134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1412/2853] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:27:27 2022] PUT /identity/v3/projects/0c212876ec464b56a48d068544e1c8c7/users/3428ab860bff4695804b00f2eb35d4ae/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:27:27.192870 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.193187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.193468 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.224166 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] MFA Rules not processed for user `3428ab860bff4695804b00f2eb35d4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:27.225323 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:27.294493 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1442/2854] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:27.306279 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:27.308034 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:27.367581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d76a3-ee3d-4c40-9461-1e4757738e52 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:27.371544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e96cc-df09-4665-a850-9f6e212fc9cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.372186 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e96cc-df09-4665-a850-9f6e212fc9cb service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.372462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e96cc-df09-4665-a850-9f6e212fc9cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.372732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e96cc-df09-4665-a850-9f6e212fc9cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.382181 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e96cc-df09-4665-a850-9f6e212fc9cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.383654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e96cc-df09-4665-a850-9f6e212fc9cb service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.393026 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1413/2855] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:27.413353 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:27.480008 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b0d0-7304-4e08-831e-b5b8607c8266 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:27.485003 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.486010 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.486299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.486577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] PATH_INFO: `/v3/users/3428ab860bff4695804b00f2eb35d4ae` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.487861 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3428ab860bff4695804b00f2eb35d4ae)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.489794 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.538001 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3428ab860bff4695804b00f2eb35d4ae'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:27.542630 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:27.554945 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3428ab860bff4695804b00f2eb35d4ae'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:27.562665 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3428ab860bff4695804b00f2eb35d4ae'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:27.570033 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3428ab860bff4695804b00f2eb35d4ae'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:27.570714 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3428ab860bff4695804b00f2eb35d4ae'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:27.576749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1443/2856] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:27:27 2022] DELETE /identity/v3/users/3428ab860bff4695804b00f2eb35d4ae => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:27.581627 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.581917 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.582222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.600254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:27.601467 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:27.667082 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1414/2857] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:27.677409 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:27.679090 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:27.744524 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1680f-7979-40d8-9d55-4ed787c28ff3 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:27.749148 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae345-47b7-4f90-9694-29b3b1471e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:27.749830 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae345-47b7-4f90-9694-29b3b1471e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:27.750122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae345-47b7-4f90-9694-29b3b1471e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:27.750517 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae345-47b7-4f90-9694-29b3b1471e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:27.761333 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae345-47b7-4f90-9694-29b3b1471e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:27.762855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae345-47b7-4f90-9694-29b3b1471e73 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:27.773995 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1444/2858] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:28.058921 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.060523 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:28.073430 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d229b8-f381-41f3-983d-78b1b593b3c0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.079150 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.080309 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.080605 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.080873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] PATH_INFO: `/v3/projects/0c212876ec464b56a48d068544e1c8c7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.082222 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c212876ec464b56a48d068544e1c8c7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.084574 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.102654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:28.143324 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c212876ec464b56a48d068544e1c8c7', 'request_id': 'req-f90f471a-2946-4a75-9f5e-06444fe863ae'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:28.149534 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c212876ec464b56a48d068544e1c8c7', 'request_id': 'req-f90f471a-2946-4a75-9f5e-06444fe863ae'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:28.156069 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c212876ec464b56a48d068544e1c8c7', 'request_id': 'req-f90f471a-2946-4a75-9f5e-06444fe863ae'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:28.169779 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] The token cache is being invalidate because project 0c212876ec464b56a48d068544e1c8c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:27:28.170050 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:28.170669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f90f471a-2946-4a75-9f5e-06444fe863ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:28.172295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1415/2859] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:27:28 2022] DELETE /identity/v3/projects/0c212876ec464b56a48d068544e1c8c7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:28.177516 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.177809 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.178096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.196882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:28.198100 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:28.261956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1445/2860] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:28.269969 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.270220 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.270529 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.290120 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:28.291393 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:28.300444 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1416/2861] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:28.305182 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.315899 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a8f22-7c90-49a3-b0a5-eb8b313a534f None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.321750 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.322458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.322760 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.323047 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.323781 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:27:28.324086 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.325749 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.339495 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1446/2862] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:27:28.343934 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.353772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431275a8-0ed1-453d-ab58-5a1ee3e6c6c0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.358904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.359600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.359935 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.360200 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.360899 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.362941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.392299 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1417/2863] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:27:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:28.395235 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.406140 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf2530-0045-4d9c-8178-9be9a4186aa9 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.466551 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.467504 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.467887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.468424 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.469389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.472080 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.479483 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:28.498569 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1447/2864] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:27:28 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:27:28.499824 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.510880 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803e0f-7dea-42ea-9c9d-a7fac92a8619 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.516926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.517526 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.517809 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.518232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.519031 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.520654 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.531876 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1418/2865] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:27:28.536291 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.547306 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48fe96-1a7a-401b-8af4-a940698f2396 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.553277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7aed4e-4d72-467a-9c59-3abf69a55265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.553987 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7aed4e-4d72-467a-9c59-3abf69a55265 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.554362 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7aed4e-4d72-467a-9c59-3abf69a55265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.554605 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7aed4e-4d72-467a-9c59-3abf69a55265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.555492 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7aed4e-4d72-467a-9c59-3abf69a55265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.557324 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7aed4e-4d72-467a-9c59-3abf69a55265 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.568278 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1448/2866] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:28.572659 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.584094 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8f2e4-b2e3-48c9-9ecb-e41068638e7a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.590025 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.590960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.591242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.591508 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] PATH_INFO: `/v3/projects/71c6f24f0b7d46c9a72fb2dd269016e5/users/141f3d75560b490893873330c5a79e77/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.607613 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c6f24f0b7d46c9a72fb2dd269016e5, user_id=141f3d75560b490893873330c5a79e77, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.610292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.625495 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1419/2867] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:27:28 2022] PUT /identity/v3/projects/71c6f24f0b7d46c9a72fb2dd269016e5/users/141f3d75560b490893873330c5a79e77/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:28.636096 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.636385 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.636659 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.664607 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] MFA Rules not processed for user `141f3d75560b490893873330c5a79e77`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:28.666015 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:28.734037 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1449/2868] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:28.750854 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.753396 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:28.824023 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7441f-bd95-4895-b9be-6142f96d1a4c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.828629 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec48842-5440-4d6a-a175-d655ccfd4fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.829295 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec48842-5440-4d6a-a175-d655ccfd4fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.829677 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec48842-5440-4d6a-a175-d655ccfd4fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.829957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec48842-5440-4d6a-a175-d655ccfd4fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.840661 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec48842-5440-4d6a-a175-d655ccfd4fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.842140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec48842-5440-4d6a-a175-d655ccfd4fe3 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:28.852141 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1420/2869] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:28.878022 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:28.945568 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df858788-48eb-41e8-8b23-c63b22e30a4b None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:28.950743 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:28.951661 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:28.951934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:28.952205 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] PATH_INFO: `/v3/users/141f3d75560b490893873330c5a79e77` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:28.953507 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=141f3d75560b490893873330c5a79e77)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:28.955855 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.005800 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '141f3d75560b490893873330c5a79e77'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.017824 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '141f3d75560b490893873330c5a79e77'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.021811 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:29.029566 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '141f3d75560b490893873330c5a79e77'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.038039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '141f3d75560b490893873330c5a79e77'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.038810 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '141f3d75560b490893873330c5a79e77'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.045185 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1450/2870] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:27:28 2022] DELETE /identity/v3/users/141f3d75560b490893873330c5a79e77 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:29.050399 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.050771 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.051054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.069910 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:29.071182 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:29.112734 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:29.137604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1421/2871] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:29.148735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:29.150243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:29.211116 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5e8e-26b2-4b47-ab7e-261ebfd55650 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:29.215363 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bca29-a8a7-4d02-bf50-0ea08aab6986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:29.216026 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bca29-a8a7-4d02-bf50-0ea08aab6986 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.216292 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bca29-a8a7-4d02-bf50-0ea08aab6986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.216564 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bca29-a8a7-4d02-bf50-0ea08aab6986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.226389 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bca29-a8a7-4d02-bf50-0ea08aab6986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:29.227872 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bca29-a8a7-4d02-bf50-0ea08aab6986 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.237671 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1451/2872] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:29.514989 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:29.516535 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:29.530428 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fad67-130d-45e7-aad1-e2202aadc708 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:29.537946 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:29.539137 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.539272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.539636 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] PATH_INFO: `/v3/projects/71c6f24f0b7d46c9a72fb2dd269016e5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.541093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c6f24f0b7d46c9a72fb2dd269016e5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:29.543544 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.596581 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c6f24f0b7d46c9a72fb2dd269016e5', 'request_id': 'req-3189449b-aaf4-406a-8e96-0599e5d4b393'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.602630 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c6f24f0b7d46c9a72fb2dd269016e5', 'request_id': 'req-3189449b-aaf4-406a-8e96-0599e5d4b393'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.610075 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c6f24f0b7d46c9a72fb2dd269016e5', 'request_id': 'req-3189449b-aaf4-406a-8e96-0599e5d4b393'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.624147 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] The token cache is being invalidate because project 71c6f24f0b7d46c9a72fb2dd269016e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:27:29.624441 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.625074 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-3189449b-aaf4-406a-8e96-0599e5d4b393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:29.626689 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1422/2873] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:27:29 2022] DELETE /identity/v3/projects/71c6f24f0b7d46c9a72fb2dd269016e5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:29.631950 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.632439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.632554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.651309 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:29.652534 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:29.719180 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1452/2874] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:29.728477 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.728842 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.729191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.749001 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:29.750226 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:29.758938 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1423/2875] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:29.764093 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:29.775891 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4eef-f9dc-4a52-9370-5cc793524066 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:29.781690 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:29.782408 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.782693 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.783102 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] PATH_INFO: `/v3/domains` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.783803 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80295) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:27:29.784141 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:29.785820 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.799528 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1453/2876] 10.222.0.64 () {58 vars in 1152 bytes} [Fri Jun 24 10:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:27:29.803846 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:29.813672 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a351ee-c9c1-4bd2-a1f3-df31c60797bd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:29.819002 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:29.819687 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.819932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.820231 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.820926 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:29.822877 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.851133 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1424/2877] 10.222.0.64 () {60 vars in 1152 bytes} [Fri Jun 24 10:27:29 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:29.855479 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:29.866481 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276c9c-ed73-4ebb-80db-616d2d23a495 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:29.923290 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:29.923956 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.924243 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.924533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] PATH_INFO: `/v3/users` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.925255 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:29.927232 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.950245 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1454/2878] 10.222.0.64 () {60 vars in 1143 bytes} [Fri Jun 24 10:27:29 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:27:29.954515 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:29.965761 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be629ccc-f957-41ab-9a87-b81c3e13e9db admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:29.970825 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:29.971497 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:29.971777 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:29.972051 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:29.972786 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:29.974402 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:29.985530 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1425/2879] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:29.989882 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:30.000631 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144be4e-4fd7-45e8-bd15-0aae8d679697 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:30.007030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fe29b-d7ab-4ecb-9bbf-337ae79faf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:30.007747 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fe29b-d7ab-4ecb-9bbf-337ae79faf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:30.007997 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fe29b-d7ab-4ecb-9bbf-337ae79faf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:30.008275 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fe29b-d7ab-4ecb-9bbf-337ae79faf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:30.008994 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fe29b-d7ab-4ecb-9bbf-337ae79faf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:30.010618 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fe29b-d7ab-4ecb-9bbf-337ae79faf8f admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:30.012156 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:30.023822 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1455/2880] 10.222.0.64 () {58 vars in 1121 bytes} [Fri Jun 24 10:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:27:30.027922 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:30.038985 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd7996-3a10-4c2d-8ceb-32beacbf296b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:30.045035 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:30.045941 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:30.046219 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:30.046473 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] PATH_INFO: `/v3/projects/4b3614fde94d47318cc97fddc060c7ba/users/1421ce8940a340f4b0e06bf3b9d4804f/roles/0f531bdb05494dcf9bf516de1fb46554` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:30.063499 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3614fde94d47318cc97fddc060c7ba, user_id=1421ce8940a340f4b0e06bf3b9d4804f, role_id=0f531bdb05494dcf9bf516de1fb46554)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:30.066423 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:30.081155 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1426/2881] 10.222.0.64 () {60 vars in 1482 bytes} [Fri Jun 24 10:27:30 2022] PUT /identity/v3/projects/4b3614fde94d47318cc97fddc060c7ba/users/1421ce8940a340f4b0e06bf3b9d4804f/roles/0f531bdb05494dcf9bf516de1fb46554 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:27:30.092142 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:30.092435 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:30.092716 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:30.120028 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] MFA Rules not processed for user `1421ce8940a340f4b0e06bf3b9d4804f`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:30.121274 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:30.190167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1456/2882] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:27:30.201191 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:30.203107 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:30.266301 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8269-37d1-42b2-bbad-1d4ace918253 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:30.270653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d418e0a-84d4-4874-94d8-185c8c19f0e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd4560d3f144b339adb62b9ddf118ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:30.271326 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d418e0a-84d4-4874-94d8-185c8c19f0e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:30.271600 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d418e0a-84d4-4874-94d8-185c8c19f0e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:30.271869 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d418e0a-84d4-4874-94d8-185c8c19f0e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:30.280795 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d418e0a-84d4-4874-94d8-185c8c19f0e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:30.282237 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d418e0a-84d4-4874-94d8-185c8c19f0e0 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:30.291855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1427/2883] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:40.062859 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:40.132097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a76300-8d06-4adc-a196-1a26a6d02db1 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:40.137097 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:40.137982 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:40.138262 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:40.138533 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] PATH_INFO: `/v3/users/1421ce8940a340f4b0e06bf3b9d4804f` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:40.139799 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1421ce8940a340f4b0e06bf3b9d4804f)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:40.141782 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:40.196303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1421ce8940a340f4b0e06bf3b9d4804f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.209233 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1421ce8940a340f4b0e06bf3b9d4804f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.217277 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1421ce8940a340f4b0e06bf3b9d4804f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.225039 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1421ce8940a340f4b0e06bf3b9d4804f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.225685 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1421ce8940a340f4b0e06bf3b9d4804f'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.231554 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1457/2884] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:27:40 2022] DELETE /identity/v3/users/1421ce8940a340f4b0e06bf3b9d4804f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:40.236785 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:40.237348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:40.237348 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:40.244525 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:40.256799 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] MFA Rules not processed for user `68ba5f09c78c41f18a7e1c6789016a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:27:40.258591 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:40.321752 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1428/2885] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:27:40.332332 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:40.334386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:40.341797 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:40.397030 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771f36-1c8a-437f-a490-c83ded3d9763 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:40.401487 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf059af1-10cf-4e93-af50-f4e628b2c7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:40.402184 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf059af1-10cf-4e93-af50-f4e628b2c7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:40.402469 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf059af1-10cf-4e93-af50-f4e628b2c7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:40.402767 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf059af1-10cf-4e93-af50-f4e628b2c7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:40.414387 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf059af1-10cf-4e93-af50-f4e628b2c7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:40.415893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf059af1-10cf-4e93-af50-f4e628b2c7e4 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:40.425988 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1458/2886] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:27:40.709822 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:27:40.711367 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:27:40.724905 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe4886-1001-4f78-9334-8e8b315a7857 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:27:40.729778 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:27:40.730680 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:27:40.730988 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:27:40.731217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] PATH_INFO: `/v3/projects/4b3614fde94d47318cc97fddc060c7ba` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:27:40.732414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b3614fde94d47318cc97fddc060c7ba)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:27:40.734387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:27:40.785540 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3614fde94d47318cc97fddc060c7ba', 'request_id': 'req-4866db8c-4adc-4919-9676-cb6184bc0c02'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.791064 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3614fde94d47318cc97fddc060c7ba', 'request_id': 'req-4866db8c-4adc-4919-9676-cb6184bc0c02'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.797691 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3614fde94d47318cc97fddc060c7ba', 'request_id': 'req-4866db8c-4adc-4919-9676-cb6184bc0c02'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.812958 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] The token cache is being invalidate because project 4b3614fde94d47318cc97fddc060c7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:27:40.813230 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.813981 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:27:40.815653 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1429/2887] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:27:40 2022] DELETE /identity/v3/projects/4b3614fde94d47318cc97fddc060c7ba => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:27:50.421415 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=24 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:27:50.809460 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:03.031303 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] REQUEST_METHOD: `POST` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:03.031908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:03.031908 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:03.060046 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.auth.core [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] MFA Rules not processed for user `5855c9996e4345aa8f71463648b6009e`. Rule list: `[]` (Enabled: `True`). {{(pid=80295) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:29:03.061258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:03.133825 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1459/2888] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:29:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:29:03.453695 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:03.455338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:03.560261 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866db8c-4adc-4919-9676-cb6184bc0c02 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:03.564254 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:03.565242 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:03.565554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:03.566089 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:03.568722 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:03.579277 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:03.580724 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:03.589639 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1430/2889] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:04.072228 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:04.073668 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:04.159139 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8811-3f46-4fda-a5d6-ec05327b2657 None None] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:04.165027 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:04.166121 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:04.166503 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:04.166735 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:04.169079 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:04.186112 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:04.187857 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:04.201796 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1460/2890] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:13.586047 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=44 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:14.196662 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=72 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:17.135415 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:17.136912 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:17.250111 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0160e5-4be0-42a3-ad72-f203d666d21a service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:17.257625 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:17.258320 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:17.258649 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:17.258960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:17.261093 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:17.275235 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:17.276855 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d9213-d1ba-4a3a-b2cd-1a5faa920a1f service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:17.288281 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1431/2891] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:20.431500 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:29:20.462085 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7c65-76ca-4bc2-a76f-d586641eacd6 service placement] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:29:20.467406 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:29:20.468082 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:29:20.468367 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:29:20.468650 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:29:20.470545 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:29:20.501024 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:29:20.503984 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:29:20.517696 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1461/2892] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:29:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:29:27.283338 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=46 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:29:30.511052 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=15 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:30:55.702284 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:55.702957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:55.702957 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:55.723554 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] MFA Rules not processed for user `5855c9996e4345aa8f71463648b6009e`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:30:55.724816 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:55.732568 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1432/2893] 10.222.0.64 () {58 vars in 957 bytes} [Fri Jun 24 10:30:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:30:56.080466 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:30:56.090218 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82ae59-0449-4695-a25c-3e54254e4c26 service placement] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:30:56.094178 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:30:56.094871 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:30:56.095157 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:30:56.095428 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:30:56.099122 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:30:56.110805 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:30:56.112923 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:30:56.124934 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1462/2894] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:30:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:31:05.716387 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=3 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:31:06.118922 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=6 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:32:21.163048 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:21.164604 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:21.178307 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a3e93-6ba4-4da8-9817-db2694f1e2e5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:21.183243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:21.183904 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:21.184189 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:21.184459 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:21.186494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:21.197858 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:21.199325 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:21.208772 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1433/2895] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:32:31.204746 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=6 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:32:35.818546 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:35.831344 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e53c13-bfd2-4227-afeb-ae988f9542c0 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:35.834065 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:35.834732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:35.835029 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:35.835300 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:35.837386 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:35.849770 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:35.851259 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:35.861371 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1463/2896] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:32:36.794830 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:36.809457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f5f4b-4424-4685-9814-ab4ead4a48e8 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:36.816978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac88daa9b11b4e13b89f3f981686ac72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:36.817781 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:36.818162 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:36.818727 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:36.820702 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:36.837457 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:36.840083 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:36.853016 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1434/2897] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:32:37.017721 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:37.028732 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6d2de-5bc1-467f-a9ba-67e077269593 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:37.033496 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:37.034299 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:37.034623 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:37.035002 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:37.037258 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:37.053289 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:37.054766 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:37.067577 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1464/2898] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:32:38.284288 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:38.295453 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e8a99-d17e-4e4f-bf29-66d85701c346 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:38.299392 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:38.300054 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:38.300327 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:38.300673 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:38.302763 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:38.317843 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:38.319339 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:38.332545 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1435/2899] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:32:47.062149 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=12 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:32:48.326669 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=12 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:32:49.519617 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:49.529586 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575950d-4a7b-40a6-8aae-04ad5a3be2f4 service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:49.533237 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad007752722c453bafd9e3a8b30226a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:49.533893 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:49.534177 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:49.534455 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:49.536315 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:49.632216 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:49.633680 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:49.653963 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1465/2900] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:32:52.878264 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:52.893990 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7651e-6226-48b7-a36e-2d2f0d72900a service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:52.901232 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:52.901932 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:52.902214 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:52.902489 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:52.904427 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:52.917570 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:52.919228 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:52.935223 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1436/2901] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:32:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:32:56.217806 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:56.220914 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:56.241713 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd491a14-6360-467c-8800-d580c9dfb2cb service neutron] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:56.246407 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:56.247364 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:56.247644 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:56.247917 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] PATH_INFO: `/v3/users/5855c9996e4345aa8f71463648b6009e` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:56.249167 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=5855c9996e4345aa8f71463648b6009e)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:56.251162 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:56.301265 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5855c9996e4345aa8f71463648b6009e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.313244 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5855c9996e4345aa8f71463648b6009e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.321298 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5855c9996e4345aa8f71463648b6009e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.329187 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5855c9996e4345aa8f71463648b6009e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.329846 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5855c9996e4345aa8f71463648b6009e'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.335898 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1466/2902] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:32:56 2022] DELETE /identity/v3/users/5855c9996e4345aa8f71463648b6009e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:32:56.529884 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:56.541243 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f0b3e-d329-4ac1-8883-ec63bda31906 service placement] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:56.615336 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:56.616208 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:56.616494 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:56.616764 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] PATH_INFO: `/v3/projects/a1af4242504b4c7fa0903c7c61b02135` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:56.618140 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1af4242504b4c7fa0903c7c61b02135)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:56.620149 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:56.667732 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1af4242504b4c7fa0903c7c61b02135', 'request_id': 'req-5c3b1cd3-459b-4b26-8766-e50e4437cc80'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.673756 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1af4242504b4c7fa0903c7c61b02135', 'request_id': 'req-5c3b1cd3-459b-4b26-8766-e50e4437cc80'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.681807 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1af4242504b4c7fa0903c7c61b02135', 'request_id': 'req-5c3b1cd3-459b-4b26-8766-e50e4437cc80'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.697909 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] The token cache is being invalidate because project a1af4242504b4c7fa0903c7c61b02135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:32:56.698134 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.698801 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.700449 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1437/2903] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:32:56 2022] DELETE /identity/v3/projects/a1af4242504b4c7fa0903c7c61b02135 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:32:56.705101 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:56.706555 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:56.728532 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb057fc-eec6-464b-9fd8-08b77c6a31ee admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:56.774887 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:32:56.803646 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:56.804642 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:56.804925 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:56.805201 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] PATH_INFO: `/v3/users/ece2d66fe73c424fbd8947ea8e927957` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:56.806473 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece2d66fe73c424fbd8947ea8e927957)` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:56.808439 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:56.858675 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece2d66fe73c424fbd8947ea8e927957'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.870538 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece2d66fe73c424fbd8947ea8e927957'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.878874 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece2d66fe73c424fbd8947ea8e927957'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.885839 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece2d66fe73c424fbd8947ea8e927957'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.886458 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.notifications [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece2d66fe73c424fbd8947ea8e927957'} {{(pid=80295) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:56.892608 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1467/2904] 10.222.0.64 () {58 vars in 1223 bytes} [Fri Jun 24 10:32:56 2022] DELETE /identity/v3/users/ece2d66fe73c424fbd8947ea8e927957 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:32:57.076319 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:32:57.078414 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:32:57.092272 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1cd3-459b-4b26-8766-e50e4437cc80 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:32:57.152873 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68ba5f09c78c41f18a7e1c6789016a58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98caa12ca1b14e52b73e3cce22837778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:32:57.153750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:32:57.154024 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:32:57.154292 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] PATH_INFO: `/v3/projects/de9aedf5d1a74ed88a6e2c013e13e849` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:32:57.155592 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de9aedf5d1a74ed88a6e2c013e13e849)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:32:57.157667 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:32:57.180750 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:32:57.217960 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de9aedf5d1a74ed88a6e2c013e13e849', 'request_id': 'req-b6903be3-fc7f-4dc6-8242-72bb06c3c836'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:57.224462 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de9aedf5d1a74ed88a6e2c013e13e849', 'request_id': 'req-b6903be3-fc7f-4dc6-8242-72bb06c3c836'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:57.231168 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de9aedf5d1a74ed88a6e2c013e13e849', 'request_id': 'req-b6903be3-fc7f-4dc6-8242-72bb06c3c836'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:57.246961 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] The token cache is being invalidate because project de9aedf5d1a74ed88a6e2c013e13e849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:32:57.247217 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:57.247841 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b6903be3-fc7f-4dc6-8242-72bb06c3c836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:32:57.250978 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1438/2905] 10.222.0.64 () {58 vars in 1232 bytes} [Fri Jun 24 10:32:57 2022] DELETE /identity/v3/projects/de9aedf5d1a74ed88a6e2c013e13e849 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:33:06.889053 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:33:07.243378 dv-cinder-847132-2-iscsi devstack@keystone.service[80296]: DEBUG dbcounter [-] [80296] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80296) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:34:28.405043 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Authenticating user token {{(pid=80295) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:34:28.406740 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:34:28.478937 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42126520-af75-42dc-83ea-b270b19dd8a2 admin admin] Validating token access rules against request {{(pid=80295) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:34:28.543779 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd302416f834b5b9b2f1a8c9955349d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7e4a8e6312d4526bed0a72bce3741b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80295) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:34:28.544451 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] REQUEST_METHOD: `GET` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:34:28.544737 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] SCRIPT_NAME: `/identity` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:34:28.545017 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80295) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:34:28.547217 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.fernet_utils [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80295) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:34:28.559877 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:34:28.561475 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf107-c7dd-48bc-ab1d-e43845829485 service placement] RBAC: Authorization granted {{(pid=80295) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:34:28.573989 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: [pid: 80295|app: 0|req: 1468/2906] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Jun 24 10:34:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:34:38.568929 dv-cinder-847132-2-iscsi devstack@keystone.service[80295]: DEBUG dbcounter [-] [80295] Writing DB stats keystone:SELECT=58 {{(pid=80295) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}